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Director's 
Message 

This  year,  the  High  Performance  Computing  Modernization 
Program  (HPCMP)  made  great  strides  in  improving  critical 
defense  capabilities  through  some  of  the  world's  most  powerful 
supercomputers.  In  2008,  the  Program  supported  leading  edge 
technology  located  at  four  Major  Shared  Resource  Centers 
(MSRCs)  and  four  Allocated  Distributed  Centers  (ADCs).  The 
Defense  Research  and  Engineering  Network  (DREN)  emerged 
as  an  important  contributor  to  combat  power  and  protection  with  the  effective  implementation  of  IPv6.  A 
funded  program  in  2008,  Computational  Research  and  Engineering  Acquisition  Tools  and  Environments 
(CREATE)  began  a  long-term  effort  to  enhance  the  productivity  of  the  Department  of  Defense  (DoD 
acquisition  engineering  workforce  by  deploying  air  vehicle,  ship,  and  antenna  design  tools.  Here,  at  the 
Program,  we  are  excited  that  the  HPCMP  continues  to  excel,  providing  the  supercomputer  technology  and 
know-how  the  DoD  scientists  and  engineers  use  every  day  to  preserve  our  nation's  resources  and  develop 
new  technologies  to  keep  America  strong. 

Dedicated  to  and  inspired  by  the  technical  community  it  supports,  the  HPCMP  promotes  collaborative 
relationships  among  scientists  and  engineers  and  mentors  and  students.  The  HPCMP  shifts  the  paradigm  of 
science  and  technology,  so  that  HPC-enabled  simulation  is  now  equal  to  developmental  live  experimentation. 
To  prepare  Americans  for  these  innovations,  the  HPCMP  conducts  outreach  programs  on  a  national  scale. 
The  Program  offers  Joint  Educational  Opportunities  for  Minorities  (JEOM)  Internships  for  those  in  the  science, 
technology  and  mathematics  majors;  University  workshops,  and  National  Defense  Science  and  Engineering 
(NDSEG)  Fellowships  for  graduate  students;  and  the  Users  Group  Conference  (UGC),  where  those  already 
involved  or  interested  in  the  Program  can  come  together  to  discuss  new  research  and  projects. 

In  2008,  the  HPCMP  continued  to  calculate  the  HPCMP's  Return  On  Investment  (ROD  for  the  DoD  in  the 
area  of  air  vehicles,  which  rely  on  the  Programs'  computing  resources.  Every  dollar  the  HPCMP  invested 
returned  at  least  seven  dollars  of  value  for  the  taxpayer. 

As  the  Program  ends  another  successful  year,  we  continue  to  improve  supercomputing  capabilities,  which 
support  the  design  and  test  of  innovative  materials  and  weapon  systems  to  save  lives  and  help  our  nation's 
Warfighters.  The  need  for  science  and  engineering  continues  to  accelerate  and  we  intend  to  develop  the 
best  solutions  for  our  nation's  defense.  Developing  the  technology  we  need  is  an  ongoing  effort— we'll 
never  be  finished,  but  the  nation  can  rest  assured  that  we  have  state-of-the-art  technology  and  a  world-class 
S&T  and  T&E  community  on  our  side. 
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Program 

Overview 


Introduction 

When  new  critical  technologies  are  needed,  where  do  Department  of  Defense  (DoD)  scientists  and  engineers  turn? 
They  turn  to  the  High  Performance  Computing  Modernization  Program  (HPCMP)  where  solving  the  hard  problems 
is  routine. 

Technological  superiority  wins  wars.  History  teaches  us  that  victory  finds  those  who  properly  prepare.  In  order  to 
mitigate  present  and  future  defense  challenges,  the  HPCMP  provides  capabilities  for  the  DoD  to  empower  a  military 
second  to  none. 


Our  military  gains  strength  through  technology.  The  DoD  HPCMP  team,  under  the  leadership  of  Mr.  Cray  Henry, 

made  major  contributions  to  the  Warfighter,  science  and  technology  (S&T),  and  the  nation. 
The  HPCMP  provides  the  supercomputer  technology  and  know-how  the  Department's 
scientists  and  engineers  use  every  day  to  preserve  our  nation's  resources 
and  develop  new  technologies  to  keep  America  strong.  The  HPCMP 
developed  a  methodology  to  calculate  the  HPCMP's  Return  On 
Investment  (ROD  for  the  DoD  in  areas,  such  as  armor/anti- 
armor  design  and  development  and  climate,  weather,  and 
ocean  (CWO)  modeling,  which  rely  on  the  Programs' 
computing  resources.  For  these  applications,  an  $821 
million  investment  yielded  returns  in  billions  of  dollars! 
Every  dollar  the  HPCMP  invested  saved  several  dollars 
for  the  taxpayer. 


HPCMP 

Mission 

Accelerate  development  and  transition 
of  advanced  defense  technologies  into 
superior  warfighting  capabilities  by 
exploiting  and  strengthening  US 
leadership  in  supercomputing, 
communications,  and 
computational 
modeling. 


High  performance  computing  (HPC)  tools  solve 
complicated  and  time-consuming  problems. 
Researchers  expand  their  toolkit  to  solve  modern 
military  and  security  problems  using  HPC  hardware 
and  software  applications.  Through  HPC  solutions, 
projects  gain  knowledge  to  protect  our  military  through 
new  weapons  systems  for  overseas  deployments  in 
Afghanistan  and  Iraq  and  assist  long-term  weather 
predictions  to  plan  humanitarian  and  military  operations 
throughout  the  world. 
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Dedicated  to  and  inspired  by  the  technical  community  it  supports,  the  HPCMP  promotes  collaborative  relationships 
among  scientists  and  engineers  and  mentors  and  students.  The  HPCMP  shifts  the  paradigm  of  science  and  technology 
with  HPC  advancements  in  simulation  now  equal  to  developmental  live  experimentation.  To  prepare  Americans 
for  these  innovations,  the  HPCMP  conducts  outreach  programs  on  a  national  scale. 

Collectively,  HPCMP  training  and  career  development  offers  opportunities  for  young  people  to  work  with  seasoned 
and  well-respected  scientists  and  engineers,  located  at  DoD  laboratories  and  test  centers  across  the  country.  With 
guidance  and  exposure  to  HPC  resources,  these  young  Americans  can  become  skilled  professionals  in  the  federal 
workforce. 

The  HPCMP's  mentorship  and  collaboration  foster  the  DoD  HPC  community  to  advance  technology  and  improve 
the  capabilities  and  effectiveness  of  the  Warfighter  and  the  DoD. 


The  HPCMP  Goals 


Commitment  to  support  the  Warfighter  and  protect  our  nation  drives  the  HPCMP  mission.  To  accomplish  this 
mission,  the  HPCMP  focuses  on  five  goals: 


Acquire,  deploy,  operate,  and 
maintain  best-value 
supercomputers 


Promote  collaborative  relationships  among 
the  DoD  computational  science 
community,  the  national  computational 
science  community,  and  minority  serving 
institutions 


Acquire,  develop,  deploy,  and  support 
software  applications  and  computational 
work  environments  that  enable  critical 
DoD  research,  development,  and  test 
challenges  to  be  analyzed  and  solved 


Continuously  educate  the  research, 
development,  test,  and  evaluation 
workforce  with  the  knowledge  to 
employ  computational  modeling 
effectively  and  efficiently 


Acquire,  deploy,  operate,  and  maintain  a 
communications  network  that  enables 
effective  access  to  supercomputers  and 
distributed  S&T/T&E  computing 
environments 


Every  component  in  the  HPCMP  organization  supports  one  or  more  goals.  The  HPCMP  tracks  and  reports  its  progress 
on  a  quarterly  and  annual  basis.  At  an  annual  joint  meeting  consisting  of  the  Configuration  Steering  Board  (CSB)/ 
Overarching  Integrated  Product  Team  (OIPT)/High  Performance  Computing  Advisory  Panel  (HPCAP)  members, 
the  HPCMP  reports  successes  and  sets  new  goals  and  priorities  for  the  upcoming  year. 
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The  HPCMP  Organization 


Components  within  the  HPCMP  organization  remain  interdependent  and  maintain  distinct  business  practices  and 
community  relationships.  Because  of  these  components,  customers  gain  state-of-the-art  tools  and  high  performance 
computing  capabilities.  High-speed  networks  reach  thousands  of  customers  in  academies,  the  Services,  Agencies, 
and  other  institutions.  HPC  Centers,  across  the  country,  experience  rapid  growth  in  computational  capabilities 
using  innovative  HPC  products. 

The  HPCMP  organizational  framework  promotes  success  and  solves  HPC  issues: 

•  HPC  Centers 
Networking 

Software  Applications  Support  (SAS) 

Resource  Management 

•  Outreach 
Oversight 


HPC  Centers 

Networking 

SAS 

Provide  supercomputers 

Provide  networks  and  advanced 
communication  capabilities 

Provide  state-of-the  art  software  tools 
to  enable  users  to  make  the  best  use 
of  HPC  hardware  resources 

Resource  Management 

Outreach 

Oversight 

Gather,  analyze,  store,  and 

Promote  HPCMP  and  foster  the 

Ensure  the  HPCMP  complies  with  US 

report  requirements,  allocation 
and  usage  information  on  DoD 
computational  projects  and  HPC 
systems  performance  to  support 
DoD/Services/Agencies  allocations, 
program  planning,  acquisition,  and 
operational  processes 

exchange  of  ideas  among  HPC 
scientific  and  computational 
communities  within  the  DoD  and  at  the 
National  level 

and  DoD  requirements 

The  HPCMP  Community 


In  the  beginning  of  FY  2008,  the  HPCMP  community  consisted  of  2,254  individual  users:  scientists,  engineers,  and 
security  experts  on  approximately  284  sites  throughout  the  United  States.  These  innovators  within  the  Air  Force, 
Army,  Navy,  and  several  defense  agencies  executed  382  projects.  In  early  FY  2009,  the  HPCMP  anticipates  significant 
growth  in  the  HPCMP  community:  4,152  users  to  execute  529  projects  at  125  sites. 

The  HPCMP  responds  to  the  computational  needs  of  its  users  and  ensures  the  appropriate  distribution  of  assets 
across  the  DoD  S&T  and  test  and  evaluation  (T&E)  communities  based  upon  documented  requirements  and  Service 
priorities.  Users  gain  access  to  high  performance  computing  centers;  for  example,  Army  Research  Laboratory  (ARL), 
Engineer  Research  and  Development  Center  (ERDC),  Naval  Oceanographic  Office  (NAVO),  Air  Force  Research 
Laboratory  (AFRL),  Maui  High  Performance  Computing  Center  (MHPCC),  Arctic  Region  Supercomputing  Center 
(ARSC),  and  other  HPC  Centers  support  several  projects,  but  not  all  users  work  in  the  military.  Some  users  from 
DoD-sponsored  project  teams  come  from  Services,  universities,  and  affiliated  agencies. 
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HPCMP  Users  in  the  Services  and  Agencies 

HPCMP's  dynamic  and  diverse  community  values  HPCMP  resources  and  the  increase  in  users  is  proof  of  the 
Program's  increasing  impact. 


Army  Participation 

ARL&ERDC 
1,343  Users 
24  Organizations 
108  Projects 
50  DREN  sites 
15  Challenge  Projects 
2  DHPIs 
5  Institutes 


Navy  Participation 

NAVO 
942  Users 
16  Organizations 
197  Projects 
40  DREN  sites 
13  Challenge  Projects 
2  DHPIs 
1  Institute 


Air  Force  Participation 

AFRL&MHPCC 
1,330  Users 
25  Organizations 
199  Projects 
21  DREN  sites 
11  Challenge  Projects 
3  DHPIs 
3  Institutes 


Agencies  Participation 

DARPA,  DTRA,  JFCOM,  MDA, 
PA&E,  and  ARSC 

537  Users 
4  Organizations 
25  Projects 
84  DREN  sites 
2  Challenge  Projects 
2  DHPIs 


FY  2008  HPC  Statistics  -  Who  are  HPC  Users? 


After  receiving  allocations  from  the  Program,  project  research  and  development  involve  Services,  Agencies,  and 
universities.  Services  and  Agencies  decide  the  amount  of  resources  to  allocate  for  each  project.  The  two  pie  charts 
below  show  user  categories  and  distribution  by  Services,  Agencies,  contractors,  and  government  personnel. 
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Computational  Technology  Areas  (CTAs) 

The  HPCMP  community  focuses  on  hundreds  of  projects  throughout  the  country,  categorized  into  areas  called 
Computational  Technology  Areas  (CTAs).  The  following  table  provides  FY  2008  CTAs  and  the  number  of  users. 


Computational  Technology  Area 


Description 


Users 


Computational  Structural  Mechanics  (CSM) 

Computational  Fluid  Dynamics  (CFD) 

Computational  Chemistry,  Biology,  and  Materials 
Science  (CCM) 


Computational  Electromagnetics  and  Acoustics 
(CEA) 


Cl innate/Weather/Ocean  Modeling  and  Simulation 
(CWO) 


Signal/lmage  Processing  (SIP) 


Covers  the  high  resolution,  multi-dimensional  modeling  of 
materials  and  structures  subjected  to  a  broad  range  of  loading 
conditions,  such  as  quasi-static,  dynamic,  electro-magnetic, 
shock,  penetration,  and  blast. 

Provides  accurate  numerical  solution  of  the  equations 
describing  fluid  and  gas  motion. 

Predicts  properties  and  simulates  the  behavior  of  chemicals 
and  materials  for  DoD  applications.  Methods  ranging  from 
quantum  mechanical,  atomistic,  and  mesoscale  modeling, 
to  multiscale  theories  that  address  challenges  of  length-  and 
time-scale  integration,  are  being  developed  and  applied.  Of 
recent  emerging  interest  in  the  CCM  CTA  are  methodologies 
that  cover  bioinformatics  tools,  computational  biology,  and 
related  areas,  such  as  cellular  modeling. 

Provides  high-resolution  multidimensional  solutions  of 
electromagnetic  and  acoustic  wave  propagation,  and  their 
interaction  with  surrounding  media. 

Involves  accurate  numerical  simulation  and  forecast  of  the 
Earth’s  atmosphere  and  oceans  on  those  space  and  time 
scales  important  for  both  scientific  understanding  and  DoD 
operational  use. 

Extracts  and  analyzes  key  information  from  various  sensor 
outputs  in  real-time;  sensor  types  include  sonar,  radar,  visible 
and  infrared  images,  signal  intelligence,  and  navigation 
assets. 


Forces  Modeling  and  Simulation  (FMS) 


Environmental  Quality  Modeling  and  Simulation 
(EQM) 


Electronics,  Networking,  and  Systems/C4l  (ENS) 


Integrated  Modeling  and  Test  Environments  (IMT) 


OTHER 


Focuses  on  the  research  and  development  of  HPC-based 
physical,  logical,  and  behavioral  models  and  simulations  of 
battlespace  phenomena  in  the  correlation  of  forces. 

Involves  the  high-resolution  modeling  of  hydrodynamics, 
geophysics,  and  multi-constituent  fate/transport  through  the 
coupled  atmospheric/land  surface/subsurface  environment, 
and  their  interconnections  with  numerous  biological  species 
and  anthropogenic  activities. 

Focuses  on  the  use  of  computational  science  in  support  of 
analysis,  design,  modeling,  and  simulation  of  electronics  from 
the  most  basic  fundamental,  first  principles  physical  level  to 
its  use  for  communications,  sensing,  and  information  systems 
engineering;  activity  ranges  from  the  analysis  and  design  of 
nano-devices  to  modeling  systems-of-systems. 

Addresses  the  application  of  integrated  modeling  and 
simulation  tools  and  techniques  with  live  tests  and  hardware- 
in-the-loop  simulations  for  the  testing  and  evaluation  of  DoD 
weapon  components,  subsystems,  and  systems  in  virtual  and 
composite  virtual-real  environments. 

Various 


423 


1,537 

501 


322 

259 


393 


78 

169 


164 


147 


159 
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Computational  Capability  at  the  Major  Shared  Resource  Centers 
(MSRCs) 

Since  2002,  the  HPCMP  steadily  increased  its  computational  capability  through  the  acquisition  of  new  systems  at 
its  MSRCs  and  Allocated  Distributed  Centers  (ADCs).  This  steady  increase  of  cutting-edge  technology  provides  the 
systems  to  support  groundbreaking  work  in  chemistry,  physics,  and  mathematics.  Through  the  Technology  Insertion 
(Tl)  process,  the  HPCMP  makes  an  annual  investment  into  the  acquisition  of  new  high  performance  computing 
systems  for  distribution  to  HPC  Centers. 


In  FY  2008,  the  HPCMP  received  approximately 
$50  million  in  total  funding  to  build  and  improve 
infrastructure,  provide  resources  to  competitive 
projects,  and  acquire  large  HPC  systems  for 
the  HPC  Centers.  The  figure  (right)  shows  the 
increase  of  computational  performance  due  to 
the  availability  of  HPC  assets  from  TI-02  to  TI-09 
(estimated)  in  total  number  of  Habus.  The  number 
above  each  bar  shows  a  factor  increase  from  the 
previous  year.  For  a  thorough  definition  of  the 
Habu  measurement,  see  page  7. 


HPC  Installations  (TI-01  Through  09)  Added  HPC  Capability  Trend 


Resource  Management 


Resource  Management  determines  the  requirements  of  HPC  users  and  reports  customer  community  special  interest 
activities,  such  as  requirements  site  visits  and  Challenge  allocation  processes.  The  Resource  Management  team 
gathers,  analyzes,  stores,  and  reports  requirements,  allocations,  and  usage  information  on  DoD  computational 
projects  and  HPC  systems  performance  to  support  DoD/Services/ Agencies  allocation,  program  planning,  acquisition, 
and  operational  decisions.  Resource  Management  consists  of  five  major  activities  that  accomplish  these  goals, 
described  in  this  section. 

Mission:  To  ensure  HPCMP  computational  resources  are  appropriately  distributed 
across  the  DoD  S&T  and  T&E  communities  based  upon  documented  requirements 

and  Service  priorities. 


Requirements 
Document  &  Analysis 


DoD  Challenge  Projects * 

29  continuing  projects 
10  new  projects 


Dedicated  HPC  Project  Investments  (DHPIs)* 
Metrics  5  continuing  projects 

Management  1  new  project 


Capability  Applications  Projects  (CAPs)* 
6  Phase  I  projects 

*The  definition  of  these  projects  are  found  on  page  9  1  Phase  II  project 

(Challenge),  page  13  (DHPI),  and  page  14  (CAPs). 


Program  Overview 


Requirements  Analysis 


To  serve  the  HPC  community,  the  HPCMP  conducts  surveys  to  capture  customer  requirements  and  continuously 
improve  processes.  HPCMP  projects  provide  their  input  in  the  Portal  to  the  Information  Environment  (plE),  an 
online  requirements  questionnaire.  In  FY  2008,  the  HPCMPO  analyzed  this  data  and  made  key  decisions,  such  as 
resource  acquisitions  and  allocations,  training  courses,  and  software  licensing.  Every  year,  the  HPCMPO  publishes 
a  requirements  analysis  report  to  discuss  survey  analysis,  which  is  available  upon  request. 

During  FY  2008,  the  HPCMP  Requirements  Team  made  site  visits  to  the  following  locations: 

US  Air  Force  Research  Laboratory,  Human  Effectiveness  Directorate  (AFRL/RH),  Brooks  AFB,  TX 
US  Air  Force  Research  Laboratory,  Materials  and  Manufacturing  Directorate  (AFRL/RX),  Tyndall  AFB,  FL 
US  Air  Force  Research  Laboratory,  Munitions  Directorate  (AFRL/RW),  Eglin  AFB,  FL 
US  Army  Armament  Research,  Development  and  Engineering  Center,  (ARDEC),  Picatinny  Arsenal,  NJ 

US  Army  Aviation  and  Missile  Research  Development  and  Engineering  Center  (AMRDEC),  Redstone 
Arsenal,  AL 

US  Army  Communications-Electronics  Command  (CECOM),  Ft.  Monmouth,  NJ 

US  Army  Medical  Research  and  Materiel  Command  (USAMRMC),  Ft.  Detrick,  MD 

US  Army  Night  Vision  Laboratory  (NVL),  Ft.  Belvoir,  VA 

Dugway  Proving  Ground  (DPG),  UT 

Naval  Surface  Warfare  Center  (NSWCC),  Carderock,  MD 

Naval  Surface  Warfare  Center  (NSWCD),  Dahlgren,  VA 

Naval  Surface  Warfare  Center  (NSWCP),  Panama  City,  FL 

Naval  Undersea  Warfare  Center  (NUWC),  Newport,  Rl 

Navy  Warfare  Development  Command  (NWDC),  Newport,  Rl 

Redstone  Technical  Test  Center  (RTTC),  Redstone  Arsenal,  AL 

Soldier  Biological  Chemical  Command  Research  Development  Engineering  Center  (SMCCOM),  Natrick,  MA 
Space  and  Missile  Defense  Command  (SMDC),  Huntsville,  AL 


Habu — a  Measure  of  Computational  Performance 

The  HPCMP  rates  computer  systems  in  terms  of  the  speed  at  which  DoD  computational  applications  run  on  the  systems. 

For  the  past  six  years,  the  HPCMP  has  run  a  suite  of  applications  on  existing  and  new  systems  to  obtain  performance 
comparisons.  By  comparing  the  timing  results  for  these  applications,  the  HPCMP  is  able  to  compare  the  performance  of 
any  system  relative  to  the  others.  In  2002,  a  1,024-processor  IBM  system  located  at  the  Naval  Oceanographic  Office  MSRC 
named  Habu,  was  designated  the  baseline  system.  Hence,  performance  measures  are  all  in  "Habu"  equivalent  units.  For 
example,  if  a  new  system  is  rated  at  two  Habus,  that  system  is  roughly  two  times  more  capable  than  a  system  rated  at  one 
Habu.  That  is,  the  new  system  executes  the  suite  of  applications  at  roughly  twice  the  performance  of  the  old.  Of  course,  any 
individual  application  may  run  faster  or  slower. 
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Performance  Metrics 


Two  types  of  system  performance  metrics  assist  the  HPCMP  to  assess  HPC  system  performance:  number  of  processor 
hours  utilized  on  each  HPC  system  by  each  user  organization  (usage)  and  average  expansion  factor  (normalized 
turnaround  time)  for  each  HPC  system. 

Together,  these  metrics  provide  an  indication  of  how  much  computational  work  each  system  performs  by  each  user 
organization  and  timeliness.  The  overall  operational  goal  is  to  balance  the  delivery  of  maximum  total  number  of 
processor  hours,  while  maintaining  an  acceptable  turnaround  time  for  users. 

Serving  S&T  and  T&E  communities,  the  HPCMP  ensures  optimum  system  performance  of  technology  across  the 
country.  The  HPCMP  tracks  the  percentage  of  system  availability  and  maintains  a  96-97  percent  cumulative  trend 
across  its  six  Centers,  ensuring  customers  benefit  from  seamless  HPC  performance  (see  figure  below). 


HPCMP  Systems  Uptime  at  the  Six  Centers  from  October  2007  to  September  2008 


Optimum  system  performance  contributes  to  customer  satisfaction.  In  the  most  recent  survey,  HPCMP  customers 
provided  favorable  scores  and  expressed  positive  feedback  regarding  HPC  resources,  as  shown  in  the  following 
chart. 


Acceptable  System  Documentation  Documentation  Documentation  Satisfied  with  Easy  to  Find 
Tum-around  Availability  is  Complete  is  Useful  is  Accurate  Help  Desk  Info  on  Centers' 

Time  Support  Web  Site 


HPC  Centers  Metrics 


Program  Overview 


HPCMP  Computational  Requirements 

HPCMP  investments  and  operational  decisions  rely  on  validated  requirements.  The  HPCMP  gathers,  analyzes,  and 
validates  user  community  requirements  to  include  systems  hardware,  software,  network,  and  training.  Furthermore, 
the  Program  takes  the  time  to  check  information  through  in-person  interviews  and  quality  assurance  checks.  Senior 
S&T  and  T&E  executives  in  each  Service  and  Agency  pay  attention  to  requirement  details  to  set  usage  priorities.  In 
FY  2008,  overall  requirements  increased  from  previous  years  and  are  projected  to  increase  at  a  steady,  consistent 
rate. 


For  FY  2008,  after  requirements 
analysis,  the  HPCMP  implemented 
an  allocation  policy  to  meet  customer 
needs.  The  HPCMP  allocates  100 
percent  of  all  of  its  resources  to  HPC 
Centers.  In  turn,  the  HPC  Centers 
allocate  these  resources  to  Services/ 

Agencies,  70-75  percent;  Challenge 
Projects,  25-30  percent;  and  other 
high-priority  projects,  1  percent. 

DoD  S&T  and  T&E  communities  that 
meet  analysis  requirements  receive  a 
spectrum  of  computational  platforms 
at  the  unclassified  and  classified  levels 
to  support  a  wide  range  of  DoD 
applications.  A  reliable  high-speed 

network  connects  users  to  powerful  resources  and  platforms  with  computational  power,  central  memory,  and  data 
storage.  HPC  balances  technology  with  aggressive  training  programs  to  educate  HPC  users  and  broaden  their 
knowledgebase.  Progress,  education,  and  innovation  optimize  the  HPC  impact  on  DoD's  ability  to  support  the 
Warfighter. 


Fiscal  Year 

Total  Computational  Requirements  of  the  HPCMP  Community 


DoD  Challenge  Projects 

The  HPCMP  allocated  approximately  30  percent  of  its  HPC  resources  in  FY  2008  to  competitively  selected  DoD 
Challenge  Projects.  A  rigorous  technical  and  mission  relevance  evaluation  determines  the  selection  of  these 
computationally  intensive,  high-priority  projects.  Senior  scientists  and  engineers  within  defense  S&T  and  T&E 
organizations,  universities,  and  industry  research  partners  head  these  high-priority  projects.  Services  and  Agencies 
allocate  the  remaining  resources  through  Service  and  Agency  unique  processes.  The  HPCMP  provides  large  blocks 
of  premium  computer  time  at  a  high-priority  and  expert  technical  assistance  to  Challenge  Projects. 

Challenge  Project  efforts  produce  and  support  key  enabling  technologies,  capabilities,  and  demonstrations  expressed 
by  the  Defense  Technology  Objectives  (DTOs).  These  enabling  DTOs,  in  turn,  support  Joint  Vision  2020  and  the 
13  Joint  Warfighting  Capability  Objectives  (JWCOs)  promulgated  by  the  Joint  Requirements  Oversight  Council  of 
the  Joints  Chiefs  of  Staff. 
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This  section  provides  a  list  of  10  new  Challenge  Projects  and  completed  Challenge  Projects  for  FY  2008.  Refer  to 
the  Appendix  for  Challenge  Projects  that  continued  into  FY  2009.  The  new  Challenge  Projects  for  FY  2008  are: 


New  FY  2008  Challenge  Projects 

Atmospheric  Turbulence  Forecasts  for  Air  Force  and  Missile  Defense  Applications 

Project  Leader:  Joseph  Werne,  Northwest  Research  Associates,  Colorado  Research  Associates  Division,  Boulder,  CO 
Sponsor:  Air  Force  Airborne  Laser  System  Program  Office 

High-Fidelity  Multidisciplinary  Simulation  of  Biologically  Inspired  Micro  Air  Vehicles  (MAV) 

Project  Leader:  Raymond  Gordnier,  Air  Force  Research  Laboratory,  Air  Vehicles  Directorate  (AFRL/RB),  Wright-Patterson 
AFB,  OH 

Sponsor:  Air  Force 

High-Resolution  Parallel  Coastal  Ocean  Modeling  for  Collaboration  on  Office  of  Naval  Research  (ONR)  and  DoD  Field 
Experiments 

Project  Leader:  Oliver  B.  Fringer,  Stanford  University,  Stanford,  CA 
Sponsor:  Office  of  Naval  Research 

Hypervelocity  Stage-Separation  for  Interceptor  Missile 

Project  Leaders:  Kevin  Kennedy,  US  Army  Aviation  &  Missile  Research,  Development  &  Engineering  Command  (AMRDEC), 
Development  &  Engineering  Center  and  Sanford  M.  Dash,  CRAFT  Tech,  Pipersville,  PA. 

Sponsor:  Army 

Multiscale  Simulations  of  High  Performance  Capacitors  and  Nanoelectronic  Devices 

Project  Leader:  Jerry  Bernholc,  North  Carolina  State  University,  Raleigh,  NC 
Sponsor:  Office  of  Naval  Research 

Numerical  Exploration  of  the  Stable  Atmospheric  Boundary  Layer  (SABL)  and  Its  Effect  on  Forecasting  Battlefield 
Weather,  Sensor  Propagation  and  Diffusion  and  Dispersion  of  Smoke  and  Other  Agents 

Project  Leader:  Patrick  A.  Haines,  US  Army  Research  Laboratory  (ARL),  White  Sands  Missile  Range,  NM 
Sponsor:  Army 

Reliable  Prediction  and  Analysis  of  3-D  Separation  Using  Computational  Fluid  Dynamics  (CFD) 

Project  Leader:  Hermann  Fasel,  University  of  Arizona,  Tucson,  AZ 
Sponsor:  Office  of  Naval  Research 

Smart  Weapon  Flight  Control  System  Design  Using  Coupled  Time-Accurate  CFD/Rigid  Body  Dynamic  (RBD)/Future 
Combat  Systems  (FCS)  Simulations 

Project  Leader:  Jubaraj  Sahu,  US  Army  Research  Laboratory  (ARL),  Aberdeen  Proving  Ground,  MD 
Sponsor:  Army 

Virtual  Prototyping  of  Directed  Energy  Weapons 

Project  Leader:  Keith  Cartwright,  US  Air  Force  Research  Laboratory,  Directed  Energy  Directorate  (AFRL/RD),  Kirtland  AFB,  NM 
Sponsor:  Air  Force 

Vulnerability  of  Structures  to  Weapons  Effects 

Project  Leader:  James  T.  Baylot,  USACE  Engineer  Research  and  Development  Center  (ERDC),  Vicksburg,  MS 
Sponsor:  Army 


In  FY  2008,  the  Challenge  Project  Proposal  Review  Board  evaluated  FY  2009  Challenge  Project  proposals.  The 
Challenge  Project  Proposal  Review  Board  reviewed  27  proposals  from  the  Services,  Defense  Threat  Reduction 
Agency  (DTRA)  and  the  Defense  Advanced  Research  Projects  Agency  (DARPA),  and  scored  proposals  based  on 
technical  merit,  computational  merit,  and  potential  for  significant  progress.  The  Deputy  Undersecretary  of  Defense 
(Science  and  Technology)  [DUSD(S&T)]  approved  41  DoD  Challenge  Projects  for  FY  2009;  17  new,  23  continuing, 
and  1  extension. 
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The  following  table  provides  a  list  of  the  1 6  Challenge  Projects  completed  in  FY  2008. 

Completed  FY  2008  Challenge  Projects 

Advanced  Chemical  Oxygen-Iodine  Laser  Technology  Development  Using  3-D  Navier-Stokes  Simulation 

Project  Leader:  Timothy  J.  Madden,  US  Air  Force  Research  Laboratory,  Directed  Energy  Directorate  (AFRL/RD),  Kirtland 
AFB,  NM 

Sponsor:  Air  Force 

The  project’s  chemical  laser  results  predicted  the  bulk  gas  flow  velocity  components  associated  with  vortex  rotation  and 
developed  a  model  for  laser-induced  fluorescence  as  a  tool  to  directly  compare  simulation  data  with  experiment  flow  imaging 
diagnostics.  The  DoD  can  benefit  from  the  optimization  of  chemical  lasers. 

Characterization  and  Prediction  of  Stratospheric  Optical  Turbulence  for  DoD  Directed  Energy  Platforms 

Project  Leader:  Frank  H.  Ruggiero,  US  Air  Force  Research  Laboratory,  Space  Vehicles  Directorate  (AFRL/RV),  Hanscom 
AFB,  MA 

Sponsor:  Air  Force 

The  project  initiated  Direct  Numerical  Simulations  (DNS)  and  Large  Eddy  Simulations  (LES)  of  gravity  wave  simulations  for 
wave  characteristics  and  Reynolds  numbers.  This  project  will  directly  support  ongoing  efforts  to  increase  the  lethality  of  the 
Airborne  Laser  (ABL)  through  improved  optical  turbulence  prediction.  Results  from  this  project  will  be  ported  to  the  ABL’s 
Atmospheric  Decision  Aid  (ADA).  Several  other  weapon,  reconnaissance,  and  communication  systems  and  acquisition 
programs  in  development  will  be  impacted  by  optical  and/or  mechanical  turbulence  and  benefit  from  this  work. 

Computational  Studies  of  Naval  SONAR  and  Non-Volatile  Random  Access  Memory  (NVRAM)  Devices 

Project  Leader:  Andrew  M.  Rappe,  University  of  Pennsylvania,  Philadelphia,  PA 
Sponsor:  Office  of  Naval  Research 

The  project’s  density  functional  theory  (DFT)  calculations  revealed  local  structure  origins  of  relaxor  behavior,  which  sets  the 
operating  range  of  SONAR  devices.  The  US  Navy  can  reap  considerable  military  advantage  with  sensitive  SONAR-detecting 
materials. 

Coupled  Aircraft/Ship  Performance  Prediction  for  Dynamic  Interface 

Project  Leader:  Susan  Polsky,  Naval  Air  Systems  Command  (NAVAIR),  Patuxent  River,  MD 
Sponsor:  Navy 

The  project  examined  all  classes  of  aircraft  (fixed-wing,  rotary-wing,  and  short-takeoff/vertical-landing  [STOVL]).  Processes 
and  data,  gathered  from  this  project,  can  enhance  manned  flight  simulators  for  dynamic  interface  test  and  evaluation 
applications. 

Coupled  Computational  Flow  Dynamics  (CFD)/Computational  Structural  Mechanics  (CSM)  Modeling  of  Structure 
Response  to  Blast  Loading 

Project  Leader:  Joseph  D.  Baum,  Science  Applications  International  Corporation  (SAIC),  McLean,  VA 
Sponsor:  Defense  Threat  Reduction  Agency 

The  project’s  simulations  model  the  complex  physical  processes  to  define  weapon-target  interaction  and  improve  DoD’s 
ability  to  protect  major  US  Government  centers,  major  transportation  links  (suspension  bridges,  tunnels,  etc.),  national 
monuments  and  infrastructure  centers  from  large  (truck-size)  terrorist  attacks. 

Design  of  Energetic  Ionic  Liquids 

Project  Leader:  Jerry  Boatz,  Space  and  Missile  Propulsion  Division  (AFRL/RZSP),  Edwards  AFB,  CA 
Sponsor:  Air  Force 

The  project  investigated  the  interactions  of  energetic  high-nitrogen  compounds  with  ultrafine  or  nanophase  aluminum 
particles.  The  DoD  can  use  results  for  potential  applications  in  rocket  and  missile  propulsion. 

Design  of  Materials  for  Laser  Protection  Applications 

Project  Leader:  Ruth  Pachter,  US  Air  Force  Research  Laboratory,  Materials  &  Manufacturing  Directorate  (AFRL/RX),  Wright- 
Patterson  AFB,  OH 
Sponsor:  Air  Force 

The  project  focused  on  theoretical  predictions  of  critical  properties  to  design  materials  that  protect  from  optical  threats, 
exhibiting  multi-photon  absorption  processes.  Proposed  computational  challenge  studies  enable  fundamental  insight  and 
prediction  for  optical  limiting  materials  developments  that  meet  specific  requirements.  The  project  significantly  impacts  an 
existing  laser  protection  program  in  the  Air  Force. 
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Completed  FY  2008  Challenge  Projects 

First-Principle  Predictions  of  Crystal  Structure  of  Energetic  Materials 

Project  Leader:  Krzysztof  Szalewicz,  University  of  Delaware,  Newark,  DE 
Sponsor:  Army  Research  Office 

The  project’s  ability  to  use  a  computational  method,  symmetry-adapted  perturbation  theory  based  on  the  density-functional 
description  of  monomers  (SATP[DFT]),  to  predict  crystal  structure  provided  accurate  results.  The  DoD  can  use  the 
SATP(DFT)  method  to  predict  properties  of  energetic  materials  and  crystals  of  other  compounds,  including  notional  materials. 

Global  Ocean  Prediction  with  HYbrid  Coordinate  Ocean  Model  (HYCOM) 

Project  Leader:  Alan  J.  Wallcraft,  US  Naval  Research  Laboratory  (NRL-SSC),  Stennis  Space  Center,  MS 
Sponsor:  Navy 

Project  results  show  that  7  km  mid-latitude  resolution  is  the  minimum  for  predication  of  the  “ocean  weather”.  The  ultimate 
goal  is  to  double  this  resolution  for  development,  evaluation,  and  investigation  of  ocean  dynamics.  HYCOM  is  planned  as 
the  next  generation  operational  global  ocean  nowcast/forecast  system  at  Navy  Oceanographic  Office  (NAVOCEANO)  and  will 
transition  from  R&D  to  NAVOCEANO. 

High  Fidelity  Electromagnetic  Target  Signatures  for  Combat  Identification  (CID) 

Project  Leader:  Mary  Ann  Gualtieri,  US  Air  Force  Research  Laboratory,  Sensors  Directorate  (AFRL/RY),  Wright-Patterson 
AFB,  OH 

Sponsor:  Air  Force 

Combat  identification  is  vital  in  the  development  of  the  US  Department  of  Defense  (DoD)  advanced  weapon  systems.  This 
Challenge  Project  focused  on  researching  key  technical  issues  for  generating  high  fidelity  electromagnetic  target  signatures 
for  CID.  This  effort  will  advance  the  DoD’s  capability  to  generate,  produce,  and  support  high  fidelity  target  signatures 
databases. 

High  Resolution  Simulation  of  Full  Aircraft  Control  at  Flight  Reynolds  Numbers 

Project  Leader:  Scott  A.  Morton,  US  Air  Force  SEEK  EAGLE  Office  (46  SK/SKI),  Eglin  AFB,  FL 
Sponsor:  Air  Force 

The  project’s  computational  methods  determine  static  and  dynamic  stability  and  control  characteristics  of  fighter  and 
transport  aircraft  with  various  store  configurations.  Services  can  benefit  from  early  discovery  of  complex  aerodynamic 
phenomena  and  these  computational  methods. 

Multidisciplinary  Computational  Terminal  Ballistics  for  Weapons  Systems 

Project  Leaders:  Kent  D.  Kimsey  and  David  S.  Kleponis,  US  Army  Research  Laboratory  (ARL),  Aberdeen  Proving  Ground,  MD 
Sponsor:  Army 

The  lethality  component  focuses  on  improving  and  developing  lightweight  and  efficient  lethal  mechanisms  for  advanced 
kinetic  energy  (KE)  ammunition,  multi-function  warheads,  and  munitions  with  scalable  lethality.  The  survivability  component 
focuses  on  innovative  protection  mechanisms,  survivability  concepts,  materials,  and  structures  applied  to  the  development 
of  mass-  and  space-efficient  protection  systems  for  troops,  equipment  and  vehicles  against  a  spectrum  of  threats. 
Characterization  of  improvised  explosive  devices  (IED)  and  the  development  of  potential  armor  solutions  to  defend  against 
these  threats  will  also  be  investigated. 

Multi-Scale  Predictability  of  High-Impact  Weather  in  the  Battlespace  Environment 

Project  Leader:  James  D.  Doyle,  US  Naval  Research  Laboratory  (NRL-MRY),  Monterey,  CA 
Sponsor:  Navy 

The  project’s  initial  results  transitioned  to  operations  in  March  2008.  This  work  will  result  in  critical  upgrades  to  the  current 
forecasting  capability  of  the  DoD  by  providing  reliable  probabilities  of  the  occurrence  of  high-impact  weather  events. 

Prediction  Capability  for  High-Speed  Surface  Ships 

Project  Leader:  Joseph  Gorski,  US  Naval  Surface  Warfare  Center,  Carderock  Division  (NSWCCD),  West  Bethesda,  MD 
Sponsor:  Navy 

The  project’s  results  demonstrate  the  ability  of  software  to  numerically  predict  vehicle  hydrodynamics.  The  US  Navy  can  reap 
benefits  from  a  computational  approach  for  the  future  design  and  analysis  of  ships. 
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Completed  FY  2008  Challenge  Projects 

Solidification  of  Complex  High  Temperature  Structural  Alloys 

Project  Leader:  Christopher  Woodward,  US  Air  Force  Research  Laboratory,  Materials  and  Manufacturing  Directorate  (AFRL/ 
RX),  Wright-Patterson  AFB,  OH 
Sponsor:  Air  Force 

Simulations  were  performed  for  elemental,  binary  and  ternary  alloys  of  Ni  with  Al,  W,  Ta,  and  Re  to  compute  equations. 
Results  establish  the  limits  of  the  accuracy  of  current  density  models  used  in  commercial  casting  software  and  offer  a  new 
path  for  systematically  improving  them. 

Statistical  Fatigue  and  Residual  Strength  Analysis  of  New  and  Aging  Aircraft  Structure 

Project  Leader:  Scott  Fawaz,  US  Air  Force  Academy,  (USAFA),  Colorado  Springs,  CO 
Sponsor:  Air  Force 

The  project  focused  on  the  development  of  advanced  computational  methods  for  reliable  fatigue  and  residual  strength 
analysis  of  bolted-riveted-bonded  metallic  structures,  such  as  full  fuselage,  wing,  and  empennage  sections.  Access  to 
analysis  results  can  reduce  inspection  requirements;  prevent  premature  retirement  of  old  aircraft;  and  increase  aircraft  safety 
for  the  US  Air  Force. 


Dedicated  HPC  Project  Investments  (DHPIs) 

Each  year,  the  HPCMP  conducts  extensive  review  of  the  department's  most  important,  high  priority,  computationally 
intensive  projects  to  determine  which  ones  require  unique,  dedicated  HPC  resources  in  order  to  meet  pertinent 
objectives  and  milestones. 

Dedicated  High  Performance  Computing  Project  Investments  (DHPIs)  are  modest-sized  HPC  systems  awarded 
to  technically  sound,  mission  critical  projects  that  cannot  be  performed  using  batch  HPC  resources  due  to 
special  operational  requirements  (e.g.,  classification  level,  real-time  response,  hardware-in-the-loop,  embedded 
implementations,  use  of  emerging  technologies). 

In  FY  2008,  three  existing  projects  that  began  in  FY  2006,  reached  completion.  See  Appendix  A  to  view  a  list  of 
the  Continuing  FY  2008  DHPIs. 


Completed  FY  2008  DHPIs 

Interactive  Algorithm  Development  for  WMD  Defense,  MIT  Lincoln  Labs,  Lexington,  MA 

This  project  focused  on  the  detection  and  decryption  of  communications  to  discriminate  targets  and  intercept  missiles. 

Operational,  Probabilistic,  Numerical  Weather  Prediction  at  High  Resolution,  Dugway  Proving  Ground,  UT 

This  project  supported  range  weather  prediction  that  provided  a  probabilistic  dispersion  of  bio-hazards. 

Virtual  Electronic  Battlefield  -  FY  2006,  US  Army  Communications-Electronics  Research,  Development  and 
Engineering  Center,  Ft.  Monmouth,  NJ 

This  project  demonstrated  a  real-time  integration  of  physical  and  simulated  network  entities  in  support  of  Command,  Control, 
Communications,  Computers,  Intelligence,  Surveillance,  and  Reconnaissance  (C4ISR)  system-of-systems  investigations  on 
network  centric  warfare. 

The  HPCMP  cancelled  one  project  due  to  poor  progress,  which  focused  on  NRL/NSWMSC's  examination  of 
contamination  prediction,  "Application  of  HPC  to  Support  Operational  Use  of  CT-Analyst".  One  of  the  two  delivered 
systems  was  never  used;  therefore,  the  HPCMP  repurposed  the  unused  system,  and  recast  the  other  system  under 
a  new  project,  "Automated  Generation  of  Nomografs  for  Urban  Chemical  Dispersion  Scenarios". 
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Although  DHPI  proposals  were  renewed,  only  one  was  awarded  in  2008: 

New  FY  2008  DHPI 


Program-restricted  Hydrocode  Calculations  of  Active,  Reactive,  and  Combined  Armor  Packages 

Project  Leader:  Robert  L.  Doney,  III,  Army  Research  Laboratory  Weapons  and  Materials  Research  Directorate  (ARL/WMRD) 
Sponsor:  Army 


This  DHPI  uses  a  2,016  core  Cray  XT5  to  test  armor  configurations  salient  to  current  and  future  conflict.  The  HPCMP 
also  funded  an  out-of-cycle  early  access  architecture  to  US  Air  Force  Research  Laboratory,  Information  Directorate, 
Rome,  NY  (AFRL/RI)  to  assess  the  capability  of  cell  processor  clusters. 

Capability  Applications  Projects  (CAPs) 

Starting  in  FY  2005,  the  HPCMP  provided  newly  acquired  systems  to  Capability  Applications  Projects  (CAPs),  to 
test  key  DoD  application  codes  on  a  substantial  portion  of  entire  HPC  systems  and  to  solve  large  problems  quickly 
by  using  a  fraction  of  the  newest,  most  capable  systems.  CAPs  goals  are: 

To  quantify  the  degree  to  which  important  application  codes  scale  to  thousands  of  processors 

To  enable  new  science  and  technology  by  applying  these  codes  in  dedicated  high-end,  capability 
environments 

The  HPCMP  focuses  on  true  capability  use  for  a  short  time  before  placing  the  systems  into  allocated  operational 
use.  System  use  is  implemented  in  two  phases: 

Exploratory  phase  designed  to  test  scalability  and  efficiency  of  application  codes  to  significant  fractions  of 
systems  (5-15  projects  on  each  system) 

Production  phase  designed  to  accomplish  significant  capability  work  with  efficient,  scalable  codes  (1  -3  projects 
on  each  system) 

In  FY  2008,  the  following  CAPs  ran  on  the  final  TI-07  acquisition,  the  ERDC  Cray  XT4,  which  arrived  in  FY  2008: 

Completed  CAPs  on  the  TI-07  Acquisition 

Advanced  Virginia-Class  Submarine  Hydrodynamic  Studies,  Other  Navy 

Atomistic  Simulations  of  Shock-Induced  Chemistry,  Detonation,  and  Sensitivity  of  Energetic  Materials,  ARO 

Characterization  and  Prediction  of  Stratospheric  Optical  Turbulence,  AFRL 

*HYCOM  Global  Ocean  with  Tides,  NRL 

^Quantum  Algorithms  for  MHD  and  Turbulent  Flows,  AFOSR 

Self-Consistent  Modeling  of  Nonequilibrium  Properties  of  Josephson  Junctions,  ONR 

*Phase  I  and  Phase  II  CAPs 


In  addition,  the  HPCMP  evaluated  10  CAP  proposals  and  awarded  six  CAPs  in  FY  2008.  These  CAPs  ran  on  the 
Navy  Cray  XT5,  IBM  P6,  and  ARSC  Cray  XT5  systems  acquired  for  TI-08.  CAPs  used  4  percent  of  total  HPC  usage 
in  FY  2008. 
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Awarded  CAPs  on  the  TI-08  Acquisition 

Anion  Exchange  Mechanism  and  the  Effect  of  Strain  on  Anion  Composition  in  Semiconductor  Superlattices 

Project  Leader:  Thomas  F.  Kuech,  University  of  Wisconsin,  Madison,  Wl 
Sponsor:  ARO 

Benchmarking  the  Optical  Lattice  Emulator 

Project  Leader:  James  Freericks,  Georgetown  University,  Washington,  DC 
Sponsor:  DARPA 

Eddy  Resolving  Modeling  of  the  Pan-Arctic  Ocean  and  Sea  Ice  (ERM-AOSI) 

Project  Leader:  Wieslaw  Maslowski,  Naval  Postgraduate  School  (NPS),  Monterey,  CA 
Sponsor:  Navy 

Large  Scale  URANS/DES  Ship  Hydrodynamics  Calculations  with  CFD  Ship-lowa 

Project  Leader:  Fred  Stern,  University  of  Iowa,  Iowa  City,  IA 
Sponsor:  ONR 

Mesoscopic  Detailed  Balance  Algorithms  for  MHD  and  Quantum  Turbulence 

Project  Leader:  George  Vahala,  The  College  of  William  and  Mary,  Williamsburg,  VA 
Sponsor:  AFOSR 

Statistical  Fatigue  and  Residual  Strength  Analysis  of  New  and  Aging  Aircraft  Structure 

Project  Leader:  Scott  Fawaz,  US  Air  Force  Academy  (USAFA),  Colorado  Springs,  CO 
Sponsor:  Air  Force 


By  the  end  of  2008,  all  TI-07  and  TI-08  CAPs  ended,  including  "Quantum  Algorithms  for  MHD  and  Turbulent 
Flows".  In  this  memory-intensive  research  project,  Dr.  George  Vahala  pushed  the  boundaries  of  quantum  research 
at  the  AFRL  MSRC.  Dr.  Vahala's  project  is  highlighted  in  the  following  success  story  on  page  1 6,  as  it  describes  his 
contributions  to  enhance  the  capabilities  of  the  DoD  to  patrol  the  US  border. 


Highlights  of  Impact  in  FY  2008 


The  HPCMPO  met  program  challenges  in  FY  2008  to  encourage  future  growth  in  the  development  of  tools  and 
new  supercomputing  capabilities  and  exceeding  customer  satisfaction.  The  following  lists  describe  the  FY  2008 
accomplishments  and  the  HPCMPO's  objectives  for  FY  2009  to  sustain  success: 


FY  2008  Accomplishments 

•  Provided  supercomputing  services  to  the  DoD  RDT&E 
community 

•  Awarded,  without  protest,  a  new $344M  technical  services 
contract  for  supercomputer  center  support 

•  Awarded,  without  protest,  contracts  for  $50M  worth  of  new 
supercomputers 

•  Developed  and  released  the  solicitation  package  for  expert 
services 

•  Established  the  “virtual”  CREATE  Program  Office 

•  Delivered  500M  processor  hours  to  the  user  community 


FY  2009  Objectives 

•  Provide  premier  services  to  the  DoD  computational  science 
and  engineering  communities 

•  Refresh  the  supercomputers  at  the  largest  supercomputing 
centers 

•  Upgrade  the  information  storage  infrastructure  across  all 
major  supercomputing  centers 

•  Award  a  new  customer  productivity  enhancement  contract 

•  Renew  the  Program’s  CNDSP  certification 

•  Develop  and  release  the  solicitation  package  for  DREN 
services 
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Massive  Quantum  Research  Project  Pushes 

Boundaries 

This  High  Performance  Computing  Modernization  Program  (HPCMP)  Capability  Applications  Project  (CAP),  “Quantum 
Algorithms  for  MHD  and  Turbulent  Flows,  ’’  was  run  at  the  AFRL  MSRC  by  Dr.  George  Vahala,  Professor  of  Physics,  College 
of  William  &  Mary,  Williamsburg,  Virginia  in  support  of  the  Hanscom  AFB.  AFRL  MSRC  System  Utilization:  SGI  Altix  4700 
(HAWK)  with  2,251,713  total  hours 


By  Gary  Sivak  and  Dinah  Luneke  from  Wright 
Cycies,  Spring  2008 

Border  Patrol  Scenario 

Guards  leisurely  glance  at  computer  screens  to  check 
on  results  from  the  continuous  scanning  for  underground 
disturbances  along  the  U.S. -Mexico  border.  In  this  future 
scenario,  thanks  to  the  field  applications  of  Jeffrey  Yepez 
(AFRL)  and  Dr.  George  Vahala’ s  work,  guards  may  more 
easily  be  able  to  detect  underground  movements  in  order  to 
prevent  illegal  aliens  from  slipping  into  the  country.  These 
and  other  future  applications  are  pushing  the  boundaries  of 
guantum  research  as  Dr.  Vahala  and  his  colleagues  work  to 
enhance  and  expand  the  capabilities  of  the  DoD 
in  order  to  serve  the  warfighter. 

HPCMP  “Tips  Its  CAP’’  for  Massive 
DoD  Projects 

Memory  can  mean  several  things  to  people 
for  a  variety  of  reasons.  Attaining  adequate 
computer  memory  for  DoD  research  projects 
elevates  this  concept  to  a  whole  new  level  for 
the  research  user  community.  In  its  quest  to  help 
DoD  researchers  fulfill  their  mission,  the  HPCMP 
provides  resources  for  large  scale  computer  runs 
for  its  CAP  program,  which  includes  goals  to: 

1.  Quantify  the  degree  to  which  important 
application  codes  scale  to  thousands  of 
processors,  including  the  identification 
of  potential  bottlenecks  for  scaling  these 
codes 

2.  Enable  new  science  and  technology  for  the 
DoD  by  applying  these  codes  in  dedicated, 
high-end,  capability  environments. 

During  the  time  period  between  acceptance 
testing  and  production  release,  the  HPCMP 
makes  select  HPC  systems  available  for  CAPs. 


Designated  users  may  test  their  application  codes  on  a 
substantial  portion  of  the  entire  system  and  solve  large, 
meaningful  problems  in  a  relatively  short  time. 

Dr.  Vahala  was  granted  a  CAP  for  the  TI-07  HAWK  system 
at  the  AFRL  MSRC  to  run  his  memory-intensive  research 
project.  A  massive  amount  of  research  was  accomplished 
through  this  9,000  processor  study.  Dr.  Vahala’s  job 
represents  the  largest  single  job  to  run  on  the  HAWK,  lasting 
almost  25  hours  and  using  224,663  processor  hours. 


v. 
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From  CFD  to  Kinetic  Velocity  Space 

Two  distinct  projects  were  part  of  this  theoretical  research  CAR 
The  first  was  a  classical  project  -  Magnetohydrodynamics 
(MHD),  similar  to  the  turbulence  found  on  Jupiter  and 
in  plasmas,  and  the  second  used  the  quantum  lattice 
approach  to  quantum  turbulence.  This  massive  computer 
run  consisted  of  homegrown  codes  that  were  mesoscopic 
detail  balanced  representations.  For  instance,  if  fluid  or 
MHD  turbulence  is  observed,  the  flow  is  collision-dominated 
and  makes  valid  the  direct  Computational  Fluid  Dynamics 
(CFD)  solution  of  the  continuum  equations  (like  the  CFD 
solution  of  the  Navier  Stokes  Equation  for  fluid  turbulence). 
According  to  Dr.  Vahala,  the  inherent  problem  with  using 
CFD  codes  for  such  large-scale  computer  analyses  is  their 
scalability  with  the  number  of  processors.  Typically,  as 
more  and  more  processors  are  added  to  solve  the  flow, 
while  working  in  a  CFD  code,  the  simulation  run  time  will 
merge  together.  In  essence,  when  doubling  the  number 
of  processors,  one  would  anticipate  that  as  the  numbers 
go  up,  the  wall  clock  time — the  time  one  has  to  wait  for 
the  simulation  results  — should  decrease.  Instead,  with 
standard  CFD  codes,  after  one  or  two  thousand  processors, 
that  will  cease.  The  more  processors  added  can  actually 
take  longer  because  the  communication  between  them 
could  result  in  a  road  jam. 

“The  codes  that  we  are  running,  instead  of  working  in 
standard  X-T  space — space/time,  go  into  kinetic  space,”  Dr. 
Vahala  said.  “That  means  more  memory,  unfortunately.  On 
the  other  hand,  the  problem  is  now  in  a  higher  dimensional 
space  and  one  can  get  a  simpler  solution  trajectory,  which 
can  be  solved  more  simply.  Not  only  that,  but  you  can 
then  parallelize  so,  in  fact,  you  do  not  see  any  saturation 
with  cores.  That’s  why  we  ran  with  all  the  9,000  cores  that 
were  available.” 

The  dimensions  used  involve  space/kinetic/velocity/  time, 
which  translates  to  a  factor  of  “6+1.”  This  simply  means, 
“Six  in  velocity-space,  and  one  in  time.”  The  advantage  of 
the  lattice  discretization  method  is  that  dimensionality  in  this 
kinetic  velocity  space  can  be  reduced.  It  is  difficult  to  store 
a  six-dimensional  array  on  a  computer  as  the  methods  are 
extremely  memory  bound. 

At  the  time  of  this  writing,  Dr.  Vahala  and  his  team  are 
“banging  at  the  data  analysis”  with  more  than  1 0  terabytes 
(TB)  of  data  from  a  single  run. 


+ 


Pushing  the  Law  of  Physics’  Boundaries 

As  this  work  progresses,  Dr.  Vahala  will  continue  to  seek 
CAP  status  at  centers  throughout  the  DoD.  Besides  running 
at  the  AFRL  MSRC,  Dr.  Vahala  has  also  run  HPC  projects 
at  NAVO  and  ERDC. 

“The  Air  Force  chap  we’re  working  with  is  viewing  his 
technique,  of  these  quantum  lattice  gas  methods,  as  being 
able  to  understand  the  law  of  basic  physics:  quantum 
chromodynamics,  quantum  electrodynamics,  even  to 
unification  with  quantum  gravity,”  Dr.  Vahala  remarked. 
“He  has  extremely  high  goals  for  this  method  using  basic 
quantum  bit  (qubit)  information  where  you  can  obtain  the 
Dirac  Equation  and  others  that  could  result  in  a  unified 
theory  spanning  all  the  fields  of  study  in  physics.” 


Small-scale  structures  in 
the  turbulent  MHD  regime 


j 
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MHD  equations  were  modeled  during  the  intensive  CAR 
These  are  the  basic  starting  equations  for  the  study  of 
turbulence  where  conducting  fluid  or  plasma  interacts  with 
a  magnetic  field.  Plasma  fusion,  solar,  magnetosphere, 
astrophysics — all  begin  with  MHD  serving  as  a  fundamental 
equation.  While  periodic  boundary  conditions  were  utilized 
in  these  CAP  runs,  the  lattice  method  readily  adapts  itself 
to  handling  arbitrary  boundary  geometries  (e.g.,  those 
generated  by  Mathcad®)  without  losing  any  of  its  excellent 
parallelization.  One  can  readily  include  more  detailed 
structures  like  yaw  grids  and  other  elements. 

Benefiting  the  Warfighter  and  the  World 

Although  it  is  still  a  little  early  to  predict  all  the  outcomes  of 
this  MHD  work,  there  are  definitely  planning  and  logistics 
benefits  expected  from  the  quantum  research.  An  area, 
such  as  the  field  of  cryptography,  works  by  understanding 
how  information  can  be  passed  along  that  cannot  be 
deciphered  without  possessing  the  encryption  key. 
Cryptography  studies  work  like  Morse  Code.  The  only 
way  to  interpret  the  message  is  to  understand  the  code.  In 
similar  fashion,  messages  sent  using  quantum  encryptions 
can  only  be  correctly  interpreted  if  one  possesses  the 
special  key  to  decipher  the  message.  This  study  of 
quantum  cryptography  can  help  make  informed  decisions 
for  potentially  intercepted  data.  The  information  would  be 
coded  in  such  a  way  as  to  be  considered  unbreakable,  and 
interception  or  eavesdropping  would  be  detectable  as  the 
interference  can  be  tested. 


The  Beat  Goes  On 


The  results  of  this  study  have  generated  a  tremendous 
amount  of  interest  in  the  scientific  community.  In  analysis, 
the  extremely  small  fluctuations  (10-14)  involve  a  study 
of  interference  patterns.  This  is  similar  to  comparing  the 
difference  between  two  musical  tones.  Normally,  one  cannot 
tell  the  difference  in  the  beats  by  simply  listening,  but  if  one 
were  to  compare  them  side  by  side,  the  differences  between 
the  two  tones  would  be  much  more  noticeable. 


“There  is  simply  no  end  in  sight  for  this  research  as  the 
work  is  quite  continuous,”  Dr.  Vahala  said.  “If  you  did 
not  have  HPC  access,  you  could  not  get  anywhere,  and 
it’s  even  stronger  than  that.  Without  HPC,  this  work  just 
could  not  be  accomplished.  Typically,  each  processor 
has  only  two  gigabytes  (GB)  of  memory.  So  to  perform 
large-scale  simulations,  one  really  needs  to  be  able  to  use 
the  9000  cores  optimally.  Our  codes  do  that.  This  type  of 
research  project  cannot  be  performed  even  with  dozens  of 
researchers  working  side-by-side  in  a  laboratory  on  PCs. 
Our  research  requires  the  enormous  computer  capability 
of  such  a  massive  system  as  the  HAWK.” 

Potential  future  applications  of  this  work  include  such  areas 
as:  improved  weather  predictions  using  advanced  ocean 
modeling;  smaller,  lighter,  and  more  efficient  engines; 
unbreakable  encryption  of  sensitive  information;  and  the 
development  of  next-generation  materials  for  submarines 
and  other  military  vehicles. 

For  more  information,  please  contact  CCAC  at  www.ccac. 
hpc.mil  or  1-877-222-2039. 
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Return  On 
Investment  (ROI) 


HPCMP  resources  are  a  fundamental  enabling  technology  for  developing  future  capabilities  and  responding  to 
immediate  combat  threats.  Over  the  past  three  years,  ROI  studies  captured  vital  statistics  and  qualitative  and 
quantitative  information  to  confirm  HPCMP  resources  and  projects  provide  high  value  for  the  Warfighter. 

ROI  History 


During  its  14-year  history,  the  HPCMP  demonstrated  its  value  through  the  development  of  Program  success  stories. 
During  2002-2003,  the  HPCMP  performed  a  User 
Impact  study  to  determine  HPCMP's  impact  on 
several  high  priority  projects,  which  was  used 
to  satisfy  Clinger-Cohen  requirements,  and 
included  an  ROI  analysis.  Recognizing  the 
importance  of  rigorously  demonstrating 
the  quantitative  value  of  the  Program  to 
the  nation's  defense  decision  makers,  the 
HPCMP  Director  established  a  pilot  project 
to  determine  ROI  for  a  subset  of  HPCMP 
projects. 


ROI 

Mission 

Ensure  that  the  value  of  HPC  is 
captured  and  successfully 
communicated  to  the 
program's  stakeholders  and 
governance  body  across  the 
Department. 


As  a  result,  ROI  analysis  delivered  valuable 
information  to  HPCMP  decision  makers. 

The  HPCMP  OIPT/CSB  recommended  the 
extension  of  ROI  analysis  to  include  formal 
presentations  at  the  annual  meeting.  In 
recent  years,  comprehensive  presentations 
at  the  annual  OIPT  meeting  provided  relevant 
qualitative  and  quantitative  benefits  of  HPCMP 
projects.  Annual  presentations  cover  ROI  activities 
and  the  quantitative  and  qualitative  benefits  derived 
from  the  studied  projects,  since  the  previous  OIPT/CSB 
meeting. 

In  FY  2008,  ROI  studies  continue  to  deliver  value  to  the  HPCMP 
and  assist  the  DoD  S&T  and  T&E  communities. 


Determine 
whether  HPCMP  is 
indeed  accelerating 
development  and  transition  of 
advanced  defense  technologies 
into  superior  warfighting 
capabilities  by  collecting 
validated  data  from  the  users 
of  information  generated  on 
our  systems. 


Develop  and 
continue  to  refine  a 
validated  and  repeatable 
process  for  determining  the 
program's  ROI,  based  on 
benefits  to  the  Warfighter,  the 
acquisition  community,  and 
research,  development,  test  & 
evaluation  (RDT&E) 
communities 
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ROI  Team 


The  ROI  team  is  comprised  of  members  from  the  HPCMPO,  the  Joint  Interoperability  Test  Command  (JITC),  and 
the  National  Defense  University  (NDU).  The  NDU  validated  the  ROI  methodology;  made  recommendations  to 
improve  future  ROI  analyses;  and  validated  results  for  individual  analysis. 

The  HPCMP  determined  each  project  investment  cost  by  allocating  a  proportion  of  the  HPCMP  total  budget  to 
each  project  based  on  the  project's  percentage  of  total  HPCMP  resource  utilization.  (For  example,  if  a  project  used 
25  percent  of  HPCMP  resources,  the  HPCMP  determined  that  the  project's  investment  cost  equaled  25  percent 
of  the  total  budget.) 

The  HPCMP  ROI  team,  HPCMPO  staff,  and  JITC,  conducted  voice,  teleconference,  and  face-to-face  interviews  with 
users  and  program  management  personnel  supporting  various  projects  using  validated  interview  questions.  After 
data  was  collected  and  analyzed,  the  ROI  team  scheduled  and  conducted  interviews  to  document  quantifiable 
benefits.  In  addition  to  documenting  quantifiable  benefits  to  the  Warfighter,  the  ROI  team  collected  qualitative 
data  and  anecdotal  "success  stories." 

The  following  lists  benefits  to  the  Warfighter  ascertained  through  ROI  studies: 


Mission  Effectiveness 

Increase  lethality 

-  Enemy  kills 

-  Enemy  equipment 

destruction 

-  Infrastructure  destruction 
Decrease  vulnerability 

-  Friendly  life  savings 

-  Friendly  equipment 
y  savings 


Better  Availability 

•  Longer  mean  time 
between  failures 
(MTBF) 

•  Shorter  mean  time 
to  repair 


Taxonomy  of  Benefits 


Test/ 

Experimentation 

Savings 


Logistics  Savings 

Fewer  spares 

Fewer  types  of 
equipments 

Reduction  in  training 
costs 

Fuel  savings 


In  FY  2008,  the  ROI  team  evaluated  the  use  of  HPCMP  resources  in  concept  development,  design,  testing,  and 
operational  support  of  Air  Vehicles. 


20 


Return  On  Investment  (ROI) 


The  following  diagram  illustrates  ROI  studies  on  air  vehicle  development  and  support  disciplines.  The  numbers  in 
each  discipline  represent  individual  projects  (Number  of  ROI-studied  projects/Number  of  studied  projects  yielding 
quantitative  results). 


Air  Vehicle  Development  &  Support  Disciplines 
Air  Vehicles 
(95/21) 


y  Test  Facilities  (16/2) 


Flying  Qualities  &  Performance  (37/6) 

•  Manned  (25/5) 

-  Fixed  Wing  (9/3) 

-  Rotary  Wing  (1 6/2) 

•  Unmanned  (5/0) 

-  Micro  Air  Vehicles  (3/0) 

-  Unmanned  Air  System  (2/0) 

•  Hypersonic  (7/1 ) 


Fatigue  (3/1) 


Compatibility  (39/12) 

•  Aircraft  Stores  (29/10) 

-  Captive  Carry  (14/4) 

-  Separation  (15/6) 


•  Aircraft-ship  (10/2) 


This  section  discusses  five  projects  from  the  ROI  studies:  Unmanned  Air  Vehicles  (UAVs)  and  EP-3E  Turret  and  Radome 
from  the  Flying  Qualities  and  Performance  category;  Aircraft/Ship  Compatibility  and  LITENING 
Pod  from  the  Compatibility  category;  Fatigue  Life  Prediction  from  the  Fatigue  category; 
and  the  Joint  Strike  Fighter  (JSF)  Sled  Test  from  the  Test  Facilities  category. 


Flying  Qualities  &  Performance 


Unmanned  Air  Vehicles  (UAV) 

In  the  late  1990s,  the  DoD  revived  the  concept  of  using  UAVs,  also  known  as  Unmanned 
Air  Systems,  for  performing  actual  combat  in  the  form  of  various  designs  generally 
designated  as  unmanned  combat  air  vehicles  (UCAVs).  Since  then,  UCAVs  played  an 
increasingly  important  role  in  the  planning  and  execution  of  military  operations.  Whether 
they  are  large  or  small,  remotely  controlled  or  fully  autonomous,  the  computational 
requirements  for  high  fidelity  design  and  airflow  modeling  require  high  performance 
computing  resources  to  avoid  costly  tests  and  experimentation.  This  section  discusses 
UAV  ROI  studies  and  the  HPCMP  impact  in  two  examples:  Unmanned  Combat  Air 
Systems  (UCAS)  Program  and  Micro  Air  Vehicles  (MAV)  Projects. 
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Unmanned  Combat  Air  Systems  (UCAS)  Program 

Aeronautical  engineering  teams  from  NAVAIR-Patuxent  River  (NAVAIR-PAX),  Arnold  AFB,  and  Wright-Patterson 
AFB  performed  researchi  projects  on  advanced  UAV  designs.  TFie  NAVAIR-PAX  team  improved  performance  of 
the  Naval  UCAS  Program  (later  recognized  as  the  Joint  Unmanned  Combat  Air  Systems  [J-UCAS])  ;  Arnold  AFB 
engineers  used  CFD  models  of  the  UCAS  to  design  improvements  to  the  craft  during  weapon  separation;  and  the 
Wrighit-Patterson  AFB  team  used  advanced  CFD  models  created  on  HPCMP  resources  to  analyze  lift,  drag,  and 
pitch  characteristics  of  various  UCAS  potential  designs. 

Thie  Air  Force's  and  Navy's  common  efforts  to  produce  an  affordable,  lethal,  low  observable  air  vehicle,  capable  of 
dynamic  distributed  control  caused  the  Services  to  merge  their  efforts,  forming  the  J-UCAS  Program.  Through  the 
use  of  HPCMP  resources  and  a  CFD  model,  the  team  analyzed  UCAS'  proposed  configuration  for  aircraft  carrier 
suitability.  The  CFD  model  enabled  the  team  to  optimize  the  configuration  for  carrier  use  during  the  conceptual 
design  phase,  saving  time  and  resources.  HPCMP  analysis  of  conceptual  UCAS  design  showed  significant  reduction 
in  drag  and  turbulence  by  increasing  the  wing  twist.  Engineers  credited  the  high  granularity  CFD  models  as  the 
only  way  to  generate  this  data,  since  flight  tests  and  wind  tunnel  data  were  unavailable.  From  their  analysis,  the 
team  helped  guide  decision  makers  to  adjust  programmatic  direction  and  build  a  knowledge  base  that  started  at 
the  beginning  of  the  program;  one  that  will  have  continued  usefulness  in  the  development  of  future  unmanned  air 
systems. 

In  addition,  a  CFD  model  was  developed  to  optimize  the  J-UCAS  flowfield  during  weapon  separation.  The  use  of 
this  model  allowed  constructing  a  test  plan  to  provide  the  optimum  combination  of  computer  simulation,  ground 
testing,  and  flight  testing.  Results  from  this  model  removed  the  need  to  move  flight  testing  to  a  larger  wind  tunnel 
or  to  design  and  build  a  smaller  model  for  the  test  chamber.  The  elimination  of  additional  tests  saved  the  Joint 
program  hundreds  of  thousands  of  dollars. 

Quantitative  benefits  for  the  J-UCAS  use  of  HPCMP  resources  were  based  on  cost  avoidance  of  not  moving  to  a  larger 
wind  tunnel.  Although  considerable  benefits  are  expected  from  the  other  work,  it  is  too  early  in  the  development 
cycle  to  quantify  these  benefits.  The  following  estimated  cost  benefits  reflect  how  HPCMP  resources  positively 
impacted  J-UCAS  research  and  development: 

J-UCAS  Cost  Avoidance  and  Savings 

Lower  bound  Upper  bound 

Cost  Avoidance  $200,000  $800,000 

Based  on  cost  of  not  moving  to  a  larger  Based  on  not  designing  &  constructing  a 
testing  wind  tunnel  smaller  testing  model 


Micro  Air  Vehicle  (MAV)  Projects 

Two  ongoing  projects  at  Wright-Patterson  AFB  used  HPCMP  resources  to  compute  flow  field  coefficients  (e.g., 
pitching  and  rolling  moment  coefficients)  to  develop  a  MAV.  All  teams  required  a  high  degree  of  fidelity  in  their 
models  to  satisfactorily  compute  these  challenging  unsteady  flow  fields.  This  type  of  computational  capability  can 
only  be  developed  with  the  support  of  high  performance  computing. 

The  development  of  MAVs  is  still  in  its  infancy.  Advanced  concepts,  such  as  flexible  or  flapping  wings,  autonomy 
of  action,  and  morphing  aircraft,  require  high  fidelity  modeling  of  fluid  and  structure,  including  nonlinear  effects. 
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More  susceptible  to  wind  currents,  MAVs  are  designed  around  non¬ 
linear  responses;  their  designs  require  a  detailed  understanding 
of  unsteady  aerodynamics  at  low  Reynolds  numbers 
(ratio  of  inertial  to  viscous  forces)  and  thij 
data  is  incorporated  into  their  designs. 

The  low  Reynolds  number  flight  regime, 
highly  flexible  structure,  and  size  and 
power  restrictions,  make  physical 
model  design  and  testing  difficult. 

Extreme  flight  conditions  demand  the 
use  of  both  nonlinear  aerodynamics 
and  structural  solvers.  Investigation  of 
dynamic  maneuvers  and  aero  servo-elastic 
problems  require  the  coupling  of  nonlinear 
aerodynamics  and  structural  solvers  with  a  flight 

mechanics  model.  Hence,  development  of  these  vehicles  cannot  be  accomplished  without  high  performance 
computing  enabled  simulation. 

The  Wright-Patterson  AFB  team  is  currently  investigating  multiple  classes  of  MAVs,  including  fixed  wing  (e.g., 
conventional,  morphing,  perching),  rotary  wing,  and  flapping  wing  models.  Because  Micro  UAVs  are  very  early  in 
their  development  cycle,  no  quantitative  benefits  are  available.  However,  Micro  UAVs  require  HPCMP  resources 
for  development,  so  this  area  is  expected  to  yield  large  quantitative  benefits  in  the  future. 

EP-3E  Program 

In  2008,  the  EP-3  Program  Office  was  tasked  with  the  integration  of  a  new  electronic  countermeasures  system 
consisting  of  a  lower  forward  turret  and  an  upper  forward  radome.  The  EP-3  Program  required  system  integration 
and  flight  clearance  to  occur  as  quickly  as  possible,  for  direct  deployment  in  active  theaters  of  operations.  Since 
the  "quick  response"  requirement  and  unavailability  of  a  suitable  subscale  model  precluded  the  use  of  wind  tunnel 
testing,  flight  engineers  decided  CFD  analysis  was  the  only  feasible  alternative. 

Researchers  divided  CFD  analysis  into  two  phases. 

The  first  phase  focused  on  the  lower  forward  turret 
and  the  second  phase  studied  the  effects  of  adding 
the  upper  forward  radome.  No  new  wind  tunnel 
testing  was  required  for  this  flight  clearance. 

Through  NAVAIR's  use  of  HPC  resources  to  perform 
CFD  modeling  and  analysis,  only  one  flying  qualities 
flight  test  was  required,  saving  three  or  more 
additional  flight  tests.  The  Computational  Research 
and  Engineering  Acquisition  Tools  and  Environments 
(CREATE)  Shadow-Operations  (Shadow-Ops) 
team  reduced  the  required  time  for  the  Navy  EP-3 
reconnaissance  aircraft's  Flying  Qualities  Flight 
Clearance  from  one  year  to  three  months  and 

greatly  accelerated  the  deployment  of  this  critical  Details  of  Modifications  from  P-3A  Patrol  Aircraft  to  EP-3E 

warfighting  asset.  The  elimination  of  subsequent 
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tests,  construction  costs,  and  flying  quality  reports  allowed  the  acquisition  program  to  meet  an  ambitious  schedule  for 
deployment  to  the  forward  theater.  Modeling  and  simulation  using  HPC  resources  saved  the  program  $1  M-$1 .4M. 
These  compressions  sped  EP-3E  fielding  and  yielded  obvious  positive  impacts  for  the  DoD  and  Warfighter. 

The  Shadow-Ops  team  used  the  EP-3E  CFD  database  in  lieu  of  test  flights  for  installation,  validation,  and  verification 
to  minimize  the  number  of  test  flights.  Because  of  the  FHPCMP's  involvement,  the  acquisition  program  saved  time, 
resources,  and  money.  The  following  list  provides  project  benefits: 

Lower  forward  turret  installation 

-  Final  flight  clearance  required  one  flight  test  only 

-  Shadow-Ops  team  saved  2-3  flights  in  Phase  I 
Upper  forward  radome 

-  No  flights  required 

-  3-4  flights  in  Phase  II  eliminated 

-  No  new  wind  tunnel  testing  required 

In  addition,  the  EP-3E  community  provided  generous  feedback  on  the  benefits  of  using  HPCMP  resources: 

"The  computational  support  supplied  by  the  Shadow  Ops  team  provided  the  required  computational 
data  to  NAVAIR  Flight  Dynamics  Branch  to  help  issue  a  safe-for-flight  clearance,  from  a  flying  qualities 
perspective,  without  the  need  for  a  costly,  time  consuming  wind  tunnel  test"  (LCDR  Ryan  Batchelor,  EP-3E 
Special  Mission  Class  Desk). 

The  data  and  subsequent  flying  qualities  analysis  was  "sufficient  to  limit  the  flight  test  program  to  only 
one  flight.  This  further  reduced  flight  test  time  by  several  weeks,  saving  the  program  tens  of  thousands  of 
dollars  and  allowing  the  EP-3E  Program  to  deploy  this  capability  in  the  forward  theater  in  support  of  the 
Global  War  on  Terror  (GWOT)"  (Ms.  Ryan  Fitzgerald,  Flight  Dynamics  Engineer). 

Without  the  use  of  F^C-enabled  modeling  and  simulation,  between  five  and  seven  flights  would  be  required  to 
obtain  flight  clearance  after  the  addition  of  the  turret  and  radome.  The  following  calculations  provide  benefits  for 
using  FHPCMP  resources: 


Lower  Bound  Cost  Avoidance  for  Certification  of  the  EP-3E 


Item 

Number  of  Test  Flights  Avoided  Cost  of  One  Test  Flight 

Total  Costs 

Turret 

Radome 

2  $200,000 

3  $200,000 

Total  Lower  Bound  Cost  Avoidance 

$400,000 

$600,000 

$1,000,000 

Upper  Bound  Cost  Avoidance  for  Certification  of  the  EP-3E 

Item 

Turret 

Radome 

Number  of  Test  Flights  Avoided  Cost  of  One  Test  Flight 

3  $200,000 

4  $200,000 

Total  Upper  Bound  Cost  Avoidance 

Total  Costs 

$600,000 

$800,000 

$1,400,000 
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Compatibility 

Aircraft-Ship  Compatibility 

Landing  an  air  vehicle  on  a  moving  ship  is  one  of  the  most  challenging  tasks 
faced  by  our  Warfighters.  Designing  ships  and  airplanes  to  accomplish  this 
task  safely  is  one  of  the  most  challenging  areas  for  the  acquisition  community. 

The  conduct  of  experiments  and  tests  to  determine  aircraft-ship  compatibility 
is  very  difficult,  so  researchers  obtain  critical  information  through  high  fidelity 
simulation.  HPCMP  resources  have  been  used  extensively  to  determine  flow 
fields  over  various  ships  for  wind  over  deck  angles  and  interaction  with  aircraft  as  it 
lands  on  ship. 

Studies  determined  airwake  for  the  following  classes  of  ships:  Amphibious 
Assault  Ship  -  Tarawa  Class  (LHA),  Amphibious  Assault  Ship  -  Wasp 
Class  (LHD),  Aircraft  Carrier  (CV),  Guided  Missile  Destroyer  (DDG), 

Multi-Mission  Surface  Combatant  Destroyer  (DDX),  and  Littoral  Combat 
Ship  (LCS). 

Using  CFD  models,  HPCMP  resources  have  been  used  to  determine  aircraft 
compatibility  with  different  types  of  ships;  the  characterization  of  airwakes  to 
calculate  safe  landing  limits,  design  of  ship  topside;  and  development  of  training 
simulators.  Specific  analysis  focused  on  solving  the  following  problems: 

Effects  of  catapulted  F-14  wake  on  H-60  operations 
Effect  of  JSF  exhaust  on  ship  and  personnel 
Limits  on  approach  and  landing  of  V-22  on  ship 

HPCMP  resources  enabled  the  Aircraft  and  Ship  acquisition  programs  to  save  an  estimated  $81.5M-$170.5M. 

The  aircraft  and  ship  communities  provided  generous  feedback  on  the  benefits  of  using  HPCMP  resources: 

"CFD-generated  airwake  data  provides  a  substantial  return  on  investment,  as  it  is  being  used  as  the  primary 
design  guidance  tool  for  USN  surface  ship  topside  design. ..the  tool  has  influenced  and  confirmed  design 
for  LHAs,  DDGs,  CVN  21,  and  CVN  28"  (Ansis  Kalnajs,  Warrant  Holder  -  Topside  Design,  Navy  Ships, 
NAVSEA). 

"[HPCMP  resources]  influenced  topside  design  on  LHA  6  and  7  [and  we]  could  not  have  gotten  the  results 
any  other  way;  possibly  wind  tunnel,  but  not  sure  it  would  have  given  correct  results.  The  alternative  would 
be  a  ship  built  with  bad  island  structure;  significant  cost  and  delay.. .huge  risk  mitigation  tool"  (Rich  Celine, 
Aviation  Systems  Engineer,  LJHA,  NAVAIR). 

"[We]  changed  island  design  on  CVN-21  as  a  result  of  testing  in  MFS  [Manned  Flight  Simulator]  with  ship 
airwake  models"  (Steve  Naylor,  NAVAIR). 

Navy  development  of  cost-effective  ship  designs  to  meet  performance  requirements  on  schedule  and  within  budget 
is  feasible  with  HPCMP  solutions. 
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The  following  table  shows  the  total  benefit  derived  from  characterization  of  the  ship  airwake  on  the  Amphibious 
Assault  Ship  -  Tarawa  Class  (LHA),  Amphibious  Assault  Ship  -  Wasp  Class  (LHD),  Aircraft  Carrier  (CV),  Guided 
Missile  Destroyer  (DDG),  Multi-Mission  Surface  Combatant  Destroyer  (DDX),  and  Littoral  Combat  Ship  (LCS).  The 
lower  bound  reflects  the  cost  of  obtaining  minimum  information  using  physical  experimentation.  The  upper  bound 
reflects  cost  to  obtain  a  useful  set  of  information  to  characterize  the  ship  airwake,  which  is  still  not  as  accurate  as 
CFD  modeling. 

Navy  Ships  Cost  Avoidance  and  Savings 

Project  Lower  Bound  Upper  Bound 

Navy  ships  $80,000,000  $160,000,000 


Litening  Pod 

LITENING  is  an  advanced  airborne  infrared  targeting  and  navigation  pod  that  makes  further  improvements  upon  the 
Advanced  Targeting  Forward  Looking  Infrared  (ATFLIR),  the  world's  first  Generation  III  Targeting  Forward  Looking 
Infrared  (FLIR).  LITENING  is  designed  to  improve  both  day  and  night  attack  capabilities  and  it  presents  pilots  with 
real-time,  FLIR  and  Charge  Coupled  Device  (CCD)  images.  It  is  fully  operational  24  hours  a  day,  including  in  adverse 
weather  conditions.  LITENING  II  can  acquire  targets  at  altitudes  of  up  to  40,000  feet,  versus  the  25,000  feet  typical 
of  previous  pods. 

The  LITENING  pod  received  multiple  benefits 
from  the  use  of  F-IPCMP  resources.  HPCMP 
resources  revealed  the  cause  of  undesirable 
bomb  separation  from  aircraft  equipped  with  the 
FLIR  and  ATFLIR.  This  analysis  showed  that  the 
blunt  aft  end  for  both  ATFLIR  and  the  LITENING 
pod  caused  a  large  shock,  which  propagates  from 
the  aft  end  to  the  adjacent  bomb  tail.  However, 
for  the  LITENING  pod,  which  is  longer  and  further 
back  on  the  aircraft,  this  shock  misses  the  tail  and 
causes  less  of  an  effect. 


Prior  to  the  LITENING  pod  effort,  the  Navy  had  two 
choices  to  clear  new  aircraft/store  configurations: 
wind  tunnel  test  or  the  buildup  approach  (also  known  as  hit  or  miss  method).  Both  methods  had  serious  limitations. 
Wind  tunnel  testing  required  at  least  six  months  of  lead-time  and  a  minimum  of  $500K.  The  buildup  approach 
consisted  of  increasing  the  release  airspeed  during  flight  testing  until  the  store  came  uncomfortably  close  to  hitting 
the  aircraft/adjacent  stores.  Because  of  the  LITENING  pod  effort,  the  Navy  now  has  a  cost-effective  and  capable 
tool  that  can  provide  a  safe  flight  test  process  to  permit  flight  clearance  recommendations  in  a  timely  fashion. 

Without  high  performance  computing-assisted  modeling,  the  LITENING  pod  equipped  aircraft  would  have  required 
operational  flight  envelope  restrictions,  similar  to  the  TFLIR/ ATFLIR  equipped  aircraft,  at  the  potential  cost  of  reduced 
mission  effectiveness  in  conflicts.  The  modeling  of  the  TFLIR  and  ATFLIR  pods  resulted  in  significant  financial  benefit 
when  the  findings  were  used  to  develop  the  LITENING  pod  within  a  two-year  schedule,  allowing  a  more  rapid 
deployment  of  advanced  technologies  to  the  Warfighter.  HPC  modeling  enabled  resources  to  focus  on  areas  of 
high  program  risk  and  provide  key  information  for  certification.  HPC  modeling  is  now  routinely  part  of  the  store 
compatibility  certification  tool  box. 


Shock  Wave  Pattern  for  ATFLIR  and  LITENING  Pod 
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HPC  modeling  helped  shorten  development  time  by  reducing  the  number  of  required  test  configurations;  reducing 
the  number  of  flight  tests  required  for  certification;  and  identifying  and  eliminating  defects  before  system  build.  Using 
data  generated  from  theTFLIR/ATFLIR  programs,  the  LITENING  pod  program  compressed  the  development  schedule 
by  two  years  and  saved  $4M.  By  avoiding  the  necessity  of  flight  tests,  the  DoD  saved  an  additional  $2M. 


LITENING  Cost  Avoidance  and  Savings 

Flight  test  costs  avoidance 

Savings  due  To  LITENING  schedule  compression 

Total  Lower  Bound  Savings 


$2,060,000 

$4,000,000 

$6,060,000 


Fatigue 


The  US  Military,  particularly  the  Air  Force  and  Navy,  has  serious  problems  with  aging  aircraft.  Air  Force  aircraft  are, 
on  average,  more  than  28  years  old— an  unprecedented  age  caused  by  the  lack  of  aircraft  procurement  during  the 
1990s  and  higher  operational  demands.  The  Services  attempt  to  stay  in  front  of  problems  from  aging  aircraft,  but 
admit  the  need  for  further  risk  mitigation. 

Military  aircraft  will  continue  to  age  dramatically.  Even  if  existing  acquisition  contracts  are  executed  as  planned  and 
the  Air  Force  and  Navy  receive  requested  fighter  aircraft,  they  will  still  have  to  field  an  unprecedented  number  of 
older  fighters,  in  order  to  meet  all  of  the  Service's  obligations.  Furthermore,  the  "operational  tempo"  for  aircraft 
operations  is  increasing,  not  decreasing  over  time. 

The  Services  devote  significant  resources  to  study  the  issue  and  pursue  innovative  solutions  to  head  off  potential 
aircraft  aging  problems.  To  meet  increased  operational  requirements,  the  Air  Force  and  Navy  take  multi-faceted 
approaches  to  manage  the  problem. 

Fatigue  Life  Prediction 

Using  F1PCMP  resources,  researchers  focus  on  the  development  of  advanced  computational  methods  for  reliable 
fatigue  and  residual  strength  analysis  of  bolted-riveted-bonded  metallic  structures.  Finite  Element  Methods  (FEM) 
are  used  to  model  large  sections  of  aircraft  at  a  very  fine  level  of  granularity  and  to  calculate  stress  intensity  factors. 
These  stress  intensity  factors  are  input  into  the  Air  Force's  crack  growth  program  (AFGROW),  to  help  predict  fatigue 
life.  Because  of  the  necessity  for  a  fine  grained  analysis  over  a  large  structure,  this  work  could  not  be  performed 
without  high  performance  computing  resources. 

The  AFGROW  with  the  FEM  option  is  currently  used  to  support  only  one  Air  Force  aircraft.  Use  of  the  AFGROW 
method  at  Hill  AFB  for  the  A-10  program  allowed  an  extension  of  inspection  intervals  by  15  percent.  This  reduction 
in  inspection  frequency  saves  the  Air  Force  $30M  annually. 

Reduced  Inspection  Savings 

Item  Benefit  Calculation  Lower  Bound  Savings 

Cost  of  7.5  fewer  inspections  per  year  7.5  inspections  x  $4, 000, 000/inspection  $30,000,000 

Total  Lower  Bound  Savings  $30,000,000 
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Test  Facilities 

Joint  Strike  Fighter  (JSF)  Tests 

The  F-35,  currently  under  development,  is  a  supersonic,  multi-role  stealth  fighter  designed  to  replace  a  number  of 
aging  fighter  and  strike  aircraft.  One  of  the  most  important  systems  for  the  JSF,  as  is  true  for  any  high  performance 
aircraft,  is  the  ejection  system.  Typically,  an  ejection  system  consists  of  rocket  powered  ejection  seat,  which  includes 
a  parachute.  This  system  is  linked  with  a  canopy  jettisoning  system  and/or  a  canopy  fracturing  capability  as  part  of 
the  ejection  seat.  Since  the  ejection  and  canopy  jettisoning  system  cannot  be  tested  in  flight,  testing  is  accomplished 
using  high  speed  sleds  which  duplicate  ejection  conditions,  as  closely  as  possible.  HPCMP  resources  were  used  to 
augment  JSF  canopy  and  ejection  seat  testing  and  ensure  that  the  testing  would  provide  useful  results. 

In  the  original  test  design,  a  high  speed  camera  assembly,  mounted  to  the  front  of  test  sled,  recorded  JSF  canopy 
images  as  the  sled  moved  down  the  track.  In  order  to  increase  the  likelihood  of  successful  tests,  researchers 
developed  computational  models  to  determine  the  effects  of  the  camera  on  the  test.  The  analysis  showed  that  the 
placement  of  the  camera  on  the  front  of  the  test  sled  would  have  invalidated  the  results  of  the  test.  In  the  second 
part  of  this  analysis,  canopy  fracturing  at  a  representative  sled  speed  was  modeled  using  an  aerodynamic  model 
combined  with  a  glass  fracturing  model.  This  analysis  showed  that  the  canopy  design  did  not  give  a  clean  breakaway, 
so  researchers  modified  the  canopy  design. 

In  addition  to  the  improvement  to  canopy  modeling,  the  CFD  research  led  to  a  design  change  in  the  pilot  seat. 
The  advanced  concept  for  the  seat  used  HPCMP  resources  to  compute  six-degrees-of-freedom  trajectories,  aiding 
rescuers  in  pinpointing  the  probable  location  of  ejection  seats  in  crash  investigations.  The  CFD  research  proved 
that  previous  testing  and  designs  did  not  produce  accurate  aeronautical  forces.  Flawed  design  could,  potentially, 
cost  the  DoD  millions  of  dollars. 

The  following  table  provides  cost  benefits  due  to  HPCMP  resources.  HPCMP  resources  enabled  cost  avoidance  in 
several  areas:  JSF  test  sled,  CFD  canopy  rupture  tests,  seat  design  improvements,  and  fore-body  design. 

JSF  Canopy  Testing  &  Seat  Redesign  Cost  Avoidance  and  Savings 
Project  Lower  Bound  Upper  Bound 

JSF  Program  office  estimate  $50,000,000  $100,000,000 


The  HPCMP's  ROI  studies  developed  and  implemented  a  validated  and  repeatable  process  for  determining  the  value 
of  the  Program  to  the  DoD  based  on  benefits  to  the  Warfighter  and  the  RDT&E  and  acquisition  communities.  This 
progress  across  all  Program  activities  optimizes  the  impact  of  HPC  on  the  DoD  S&T  and  T&E  programs'  support  of 
the  warfighting  mission  now  and  in  the  coming  years. 
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HPC  Centers  provide  support  to  solve  demanding  computational  problems  and  develop  innovative  products  for  the 
Warfighter.  Pioneering  DoD  scientists  and  engineers  throughout  the  United  States  use  the  Major  Shared  Resource 
Centers  (MSRCs)  and  Allocated  Distributed  Centers  (ADCs)  to  access  leading  edge  technology  and  expertise  across 
a  range  of  HPC  services: 


HPC 
Centers 
Mission 

To  deliver  world-class  commercial, 
high-end,  high  performance 
computational  capabilities  to  the 
DoD's  science  and  technology 
and  test  and  evaluation 
communities. 


Supercomputers 
High-Speed  Networks 
Mass  Storage 
Data  Exploration 
Customer  Assistance 

Infrastructure  and  Operations  Management 

In  FY  2008,  MSRCs  and  ADCs  provided 
resources  and  made  improvements  to  continue 
user  support. 

Major  Shared 
Resource  Centers 
(MSRCs) 


The  MSRCs  are  large  HPC  Centers  that  deliver 

leading-edge,  high  performance  computational 

resources  and  expertise  to  DoD  scientists  and  engineers 

within  the  DoD  Services  and  Agencies,  anywhere  in  the 

USA.  The  MSRCs  provide  DoD  with  "purple"  support,  allowing 

the  Air  Force,  Army,  Navy,  and  DoD  Agencies  to  share  access  to  and 

use  of  HPC  resources. 


In  FY  2008,  four  MSRCs  supported  the  defense  community: 

US  Air  Force  Research  Laboratory  (AFRL)  MSRC,  Wright-Patterson  AFB,  Dayton,  OH 

US  Army  Corps  of  Engineers  (USACE),  Engineer  Research  and  Development  Center  (ERDC)  MSRC,  Vicksburg, 
MS 
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US  Army  Research  Laboratory  (ARL)  MSRC,  Aberdeen  Proving  Ground,  MD 
Naval  Oceanographic  Office  (NAVO)  MSRC,  Stennis  Space  Center,  MS 

Through  its  annual  Technology  Insertion  (Tl)  process,  the  HPCMP  delivers  and  updates  supercomputing  capability 
at  its  HPC  centers  (See  Technology  Insertion  section).  Often,  these  new  HPC  systems  stress  power,  cooling,  weight, 
and  space  available  at  the  HPC  Centers.  In  conjunction  with  the  host  organization,  the  HPCMP  supports  the  costs 
of  facility  enhancements  to  meet  customer  needs. 

In  FY  2008,  the  HPCMP  upgraded  and  refurbished  MSRCs,  in  a  continuing  effort  to  provide  state-of-the-art  equipment 
and  technology.  The  following  list  provides  details  for  major  upgrades  and  improved  facilities  to  accommodate  HPC 
resources: 

AFRL  MSRC  installed  a  power-distribution  switch  to  receive  incoming  power  from  two  sources  and  began  the 
planning  process  for  a  new  building. 

ARL  MSRC  converted  its  facility  to  maintain  the  new  HPC  system  for  classified  use. 

ERDC  MSRC  gained  a  new  building  to  house  the  power  distribution  and  power  back-up  assets. 

NAVO  MSRC  refurbished  its  entire  computer  floor  to  facilitate  one  of  their  new  HPC  systems  and  acquired  new 
computer  floor  space  in  another  building  for  a  new  system  and  existing  systems. 

Collectively,  the  MSRCs  have  16  large  HPC  systems,  of  which,  a  few  of  the  larger  are  two  8,000+  core  Cray  Inc;  XT 
systems,  a  9,216  core  SGI  system;  and  a  5,312  core  IBM  system.  In  FY  2008,  the  MSRCs  operated  systems  totaling 
402  teraFLOPS  (one  teraflop  equals  1  trillion  Floating  point  Operations  Per  Second)  of  computational  capability. 

Allocated  Distributed  Centers  (ADCs) 

The  ADCs  are  partner  HPC  Centers  that  supplement  the  computational  capacity  of  the  MSRCs.  Although  they  offer 
less  HPC  variety  or  capacity  than  the  MSRCs,  they  provide  a  significant  additional  total  capability  to  HPCMP  customers. 
In  addition,  each  supports  a  specialized  community  from  either  the  DoD  or  academia.  ADCs  include: 

Arctic  Region  Supercomputing  Center  (ARSC),  Fairbanks,  AK 

Army  High  Performance  Computing  Research  Center  (AHPCRC),  Moffett  Field,  CA 

Army  Space  and  Missile  Defense  Command  (SMDC),  Huntsville,  AL 

Maui  High  Performance  Computing  Center  (MHPCC),  Air  Force  Maui  Optical  and  Supercomputing  Site 
(AMOS),  Kihei,  HI 

Collectively,  the  ADCs  have  several  large  HPC  systems,  including  a  2,320  core  SUN  system  at  ARSC  and  a  5,120 
core  DELL  system  at  MHPCC.  In  FY  2008,  the  ADCs  operated  systems  totaling  64  teraFLOPS  of  computational 
capability. 


Customer  Focus 


Striving  to  meet  the  needs  of  customers,  the  HPC  Centers  use  a  combination  of  synchronized  resource,  workload, 
and  enterprise  system  management  services.  An  array  of  advanced  HPC  technologies,  resources,  and  service  support 
enables  scientific  productivity  by  HPCMP  customers.  This  year,  with  the  advent  of  a  single  support  contract  for  the 
four  MSRCs,  the  Consolidated  Customer  Assistance  Center  (CCAC)  implemented  a  streamlined  set  of  procedures 
to  speed  up  trouble  ticket  resolution— 95  percent  of  700  tier  1  customer  call  tickets/ month  was  resolved  in  less  than 
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90  minutes  per  request!  The  Data  Analysis  and  Assessment  Centers  (DAACs)  provided  the  ezViz  and  Computational 
Support  Environment  (CSE)  analysis  tools  to  support  a  wide  variety  of  unclassified  and  classified  projects.  HPCMP 
projects  made  over  600,000  self-serve  requests  via  these  tools  during  2008. 

In  addition,  DAAC  personnel  directly  collaborated  with  users  1 60  times  to  provide  data  analysis  services.  Application 
Support  Analysts  work  directly  with  users  to  enhance  their  productivity  by  optimizing  applications  for  use  on  HPC 
systems.  Through  collaborative  working  relationships,  analysts  support  users  with  advanced  use  of  compilers, 
message  passing  libraries,  and  performance  analysis  tools.  Both  in-house  government  applications  and  commercial 
applications  are  supported.  Behind  the  scenes,  infrastructure  and  operations  management  activities  include  system 
and  network  administration,  security  management,  facilities  planning  and  enhancement,  and  24X7  operational  and 
performance  monitoring.  This  support  allows  scientists  and  engineers  to  focus  on  the  computational  science  instead 
of  the  infrastructure  required  to  support  their  computing  needs. 

The  HPCMP  assists  customers  through  collaboration,  technology,  and  service  support. 

Consolidated  Customer  Assistance  Center  (CCAC) 

As  the  first  point  of  contact  for  HPC  users  that  need  customer  service,  CCAC  is  accessible  through  telephone,  Web 
pages,  and  email. 

Data  Analysis  and  Assessment  Centers  (DAAC) 

Providing  specialized  data  analysis  to  customers,  DAAC  supports  both  unclassified  and  classified  research  projects. 

Application/System  Performance  Analysis 

Code  Optimization  and  Code  Porting  are  two  of  the  areas  where  Centers’  specialists  directly  support  users  to  make 
their  jobs  run  faster  and  more  efficiently. 

Infrastructure  &  Operations  Management 

Behind  the  scenes  support  for  systems  and  network  administration,  security  management,  and  infrastructure 
configuration  management  enables  scientists  and  engineers  to  focus  on  the  computational  science  instead  of  the 
infrastructure  needed  to  compute. 

Cross-Center  Projects 

Many  projects  require  collaboration  and  cooperation  between  the  HPC  Centers.  HPC  Centers  deliver  a  rich  set  of 
services,  leveraging  the  unique  expertise  of  each  HPC  Center  to  accomplish  a  common  goal  of  improving  DoD's 
computational  capabilities. 

The  following  cross-center  projects  pursued  in  FY  2008  provide  increased  computational  capability,  enhanced 
data  storage  and  migration,  improved  software  management  and  distribution,  and  improved  common  operating 
environments  and  other  tools  beneficial  to  scientists  and  engineers. 

Cross-center  projects  include: 

Technology  Insertion  (Tl) 

Storage  and  Lifecycle  Initiative 
Consolidated  Software  Initiative 
User  Interface  Toolkit  (UIT) 

Baseline  Configuration 

Next  Generation  Technical  Services  (NGTS)  Contract 
Enterprise  System  Monitoring  (ESM) 
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Technology  Insertion  (TI) 

Each  year,  the  Centers  collaborate  to  determine  the 
optimum  set  of  H  PC  systems  to  acquire.  The  following 
pie  chart  illustrates  the  teraFLOPS  (1  trillion  Floating 
point  Operations  Per  Second)  capacity  increase  due 
to  new  FY  2008  HPC  system  acquisitions.  The  increase 
in  teraFLOPS  corresponds  to  the  ability  to  do  more 
computational  work  or  to  apply  more  processing 
power  to  important  problems,  thus  reducing  the 
overall  time  to  achieve  a  solution. 

At  the  beginning  of  FY  2008,  the  HPC  systems  at  the 
four  MSRCs  and  ADCs  had  a  total  computational 
capability  of  437  teraFLOPS  (the  capability  to  perform  437  trillion  mathematical  operations  per  second).  During 
FY  2008,  the  HPCMP  procured  large  systems  for  deployment  with  a  computational  capability  of  341  teraFLOPS, 
of  which,  90  were  installed  in  2008  and  251  will  be  installed  in  early  2009.  Therefore,  the  total  capability  of  the 
HPC  systems  at  the  four  MSRCs  and  ADCs  increased  from  437  to  527  teraFLOPS.  Once  the  remaining  systems  are 
installed  in  early  FY  2009,  the  total  will  reach  778  teraFLOPS. 

Storage  &  Lifecycle  Management  Initiative 

HPCMP  customers  perform  research,  design,  and  test  activities  at  the  HPC  Centers  that  produce  prodigious  amounts 
of  data  to  document  and  support  analyses.  HPC  Centers  provide  a  high-performance,  high-capacity,  and  highly- 
reliable  mass  storage  capability  to  retain  these  data  files,  as  long  as  necessary.  As  the  available  supercomputing  power 
to  RDT&E  users  increases,  users  can  perform  larger  and  more  complex  analyses,  providing  better  understanding  and 
technology  for  the  Warfighter  of  today  and  tomorrow.  As  the  analyses  increase  in  scope  and  size,  so  do  associated 
data  files.  These  files  become  harder  to  use  and  store. 

At  the  forefront  of  technology,  in  early  FY  2008,  the  HPC  Centers  initiated  a  plan  to  provide  improved  user  productivity 
tools  to  enable  management  of  the  increasing  amount  of  data  storage  forecast  over  the  next  10  years.  The  plan's 
objective  is  to  design  and  implement  a 
HPCMP  next  generation  data  storage 
architecture  critical  to  provide  continued 
support  for  DoD  R&D  efforts.  The  plan 
focuses  on  management  of  RDT&E  data 
files  over  their  entire  lifecycle.  This  plan  will 
provide  improved  tools  to  allow  customers 
to  categorize  their  data  for  later  access 
and  use. 

The  initiative  supports  automated  data 
migration,  user  data  management  tools, 
and  disaster  recovery  for  HPC  Centers' 
systems. 

Storage  Lifecycle  Management  is  the 
fundamental  new  capability  that  enables 


Adaptable 


Data 

Management 

Tools 


Storage 

Visibility 


Next  Generation 
Storage  Architecture 

Archival 

Storage 

HPC  Centers’  next  generation  storage  architecture  leverages  technology  advances 
and  product  integration  to  address  critical  requirements  for  success  in  managing 
future  data  storage  and  data  management  requirements.  The  initiative  supports 
automated  data  migration,  user  data  management  tools,  and  disaster  recovery  for 
HPC  Centers’  systems. 


UFY2008  Baseline 


h  Installed  New  FY2008 
HPC  Systems 

n  Acquired  New  FY  2008 
HPC  Systems  to  be 
Installed  Early  FY  2009 


TeraFLOPS  of  New  HPC  Systems  for  FY  2008 
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the  next  generation  of  HPCMP  data  storage.  It  integrates  Information  Lifecycle  Management  techniques  with  storage 
infrastructure  and  will  enable  HPCMP  customers  to  assign  metadata  attributes  to  data  files  for  file  identification  and 
management.  This  capability,  designed  during  2008,  will  be  procured  during  2009,  with  initial  operating  capability 
at  the  HPC  Centers  in  late  2009. 

The  metadata  attributes  will  be  used  for  new,  more  powerful,  user  tools  to  find  and  manage  data  sets  and  provide 
information  for  the  data  archival  and  disaster  recovery  infrastructure  that  operates  behind  the  scenes.  Disaster 
recovery  infrastructure  preserves  and  protects  the  research  data  generated  by  HPCMP  customers.  The  additional 
power  provided  by  the  metadata  attributes  will  enable  users  to  gain  more  access  to  and  control  data  files,  enabling 
significantly  improved  lifecycle  management. 


Consolidated  Software  Initiative 

The  capability  to  share  commercial  software  licenses  across  HPC  Centers  and  platforms  resulted  in  significant  cost 
savings  for  the  Program  and  its  user  community.  2008  marks  the  evolution  of  this  capability  from  addressing  S-only 
commercial  applications  software  to  a  more  comprehensive  software  management  process  that  addresses  all  HPC 
applications  software  and  tools  at  the  Centers.  Now,  HPC  Centers  have  even  greater  flexibility  to  provide  more 
user  functionality,  while  realizing  cost  savings. 

User  Interface  Toolkit  (UIT) 

The  HPCMP  provides  the  UIT  as  a  resource  for  developers  to  build  user-friendly  desktop  applications  to  access 
HPCMP  resources.  The  UIT  is  an  Application  Program  Interface  (API)  that  provides  for  secure  authentication  to 
access  HPCMP  resources  and  commands  through  a  single  library.  Applications  that  use  UIT  and  its  complete  job 
stream  management  capability,  allow  novices  and  experts,  alike,  to  manage  workflows.  The  second  major  version 
of  this  set  of  tools,  with  streamlined  interfaces  and  increased  capabilities,  was  released  in  2008.  Numerous  Web- 
based  training  sessions  and  four  seminars  are  available  to  current  and  potential  new  users  and  developers. 


Baseline  Configuration 


The  Baseline  Configuration  initiative  ensures  that  a  common  and  user-friendly  operating  environment  is  available 
across  the  diverse  supercomputer  and  support  systems  within  the  HPCMP.  HPC  users  can  seamlessly  adapt  to 
the  environments  at  each  HPC 


Center  because  of  the  common 
set  of  capabilities  and  functions  for 
HPC  systems.  The  HPC  Centers 
concentrate  on  three  aspects  of 
operation:  common  software 
environment,  standardized 
processes,  and  user  support 
features. 

This  year,  the  team  added  four 
new  policies  to  the  19  previously 
developed  policies  and  made  five 
major  revisions  to  older  policies  to 
keep  the  baseline  configuration 


Common  Software  Environment 

Open  Source  Libraries 
Utilities  and  Compilers 
Editors 

Scripting  Tools 
Environment  Variables 
Debuggers 
Login  Shells 

Multi-version  Software  Modules 


Standardized  Processes 

Queue  Names 
User  Notices 
Security  Access 


User  Support  Features 

Cross  Center  Transfer 
Sample  Code  Repository 
\Ticket  Life  Consistency 


Baseline  Configuration  concentrates  on  three  aspects  of  HPC  Center  operations:  A  common 
software  environment,  standardized  processes,  and  user  support  features. 
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current  with  the  needs  of  the  user  community.  The  common  baseline  across  systems  enabled  some  customers  to 
use  as  many  as  eight  different  HPC  systems  during  2008  and  to  focus  on  their  R&D  mission  instead  of  requiring 
them  to  revise  software  and  support  tools  to  make  them  compatible  with  different  environments. 

Next  Generation  Technical  Services  (NGTS)  Contract 

In  a  continuing  effort  to  provide  more  efficient  and  effective  services  at  the  MSRCs,  the  Program  restructured  its 
support  contracts  at  the  four  largest  centers,  creating  a  single  contract  activity  in  FY  2008,  the  NGTS  contract.  The 
F1PC  Centers  collaborated  with  the  General  Services  Administration  (GSA)  to  restructure  the  technical  services 
contract  and  consolidate  four  MSRC  contracts  into  one  enterprise  approach  focused  on  standardizing  delivery  and 
achieving  common  goals. 

On  March  19,  2008,  Lockheed  Martin  was  awarded  the  NGTS  contract,  a  five-year,  $344M  contract  to  operate  the 
MSRCs  and  support  F1PCMP  assets,  programs,  and  specialized  F1PC  operations  funded  by  services  and  laboratories. 
At  no  cost  to  the  government,  F1PC  Centers  made  a  seamless  60-day  transition  to  the  NGTS  contract.  The  NGTS 
contract  supports  30-50  full-time  employees  per  F1PC  Center  and  facilitates  leveraging  innovations  and  efficiencies 
across  multiple  F1PC  Centers.  A  single  contractor  and  contract  structure  provides  significant  opportunities  for 
increased  collaboration  and  synergy  among  MSRCs  to  reduce  costs  and  delivery  times  and  to  propagate  best 
practices  across  the  Program. 

Enterprise  System  Monitoring  (ESM) 

In  2008,  one  of  the  NGTS  contract  initiatives  focused  on  providing  centralized  monitoring  of  F1PC  systems,  support 
systems,  and  network  devices  to  improve  system  management  and  customer  assistance.  This  year,  the  Program 
completed  definition  of  scope  and  coordination  with  all  potential  stakeholders  to  enable  initiation  of  detailed 
capabilities  definitions  and  system  design. 
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Defense  Research  and  Engineering  Network  (DREN) 
and  Security 


No  matter  where  you  are  in  the  country,  HPC  users  have  the  ability  to  link  to  high  performance  computing 
resources  through  the  Defense  Research  and  Engineering  Network  (DREN).  Nationwide,  DREN  connects  HPCMP's 
geographically  dispersed  E-l PC  networks.  Military  services  and 
associates  gain  secure,  robust  access  to  supercomputers 
to  collaborate  with  other  high-end  computing  users 
in  the  defense  community  through  the  DREN.  The 
Missile  Defense  Agency  (MDA),  DoD  Modeling 
&  Simulation  Office,  Joint  Forces  Command,  and 
Defense  Threat  Reduction  Agency  all  collaborate 
on  HPC  networks  for  R&D  and  T&E  programs  to 
benefit  the  Warfighter. 


DREN 

Mission 

Manage  the  Defense  Research  and 
Engineering  Network  (DREN)  as 
DoD's  recognized  research  and 
engineering  network. 


Today,  we  have  flexibility  to  enable  customers 
in  remote  regions  of  the  country  to  access 
HPC  resources.  The  DREN  is  an  enabler 
that  expands  the  pool  of  HPC  researchers 
available  to  the  DoD  to  solve  the  hard 
problems. 

Increasing  the  pace  of  federal  collaborations, 
the  DREN  contributes  to  overall  federal  agency 
networking  and  security  through  the  Large 
Scale  Network  (LSN)  and  Joint  EngineeringTeam 
(JET).  As  elements  of  the  White  House's  Office 
of  Science  and  Technology  Policy  Interagency 
Working  Group,  LSN  and  JET  extend  US  technological 
leadership  in  leading-edge  network  technologies  and 
coordinate  federal  agency  networking  activities,  operations, 
and  plans. 


Security 

Mission 

Protect  information  with  a 
security-in-depth  model  that 
includes  access  filters,  strong 
authentication,  active 
monitoring,  and 
encryption. 
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In  2008,  DREN  exceeded  its  target  availability  goal  of  99.8%,  providing  connectivity  to  customers  throughout  the 
country. 


Month  Target  Availability  >99.8% 


Jan-08 

99.967% 

Feb-08 

99.963% 

Mar-08 

99.961% 

Apr-08 

99.978% 

May-08 

99.969% 

Jun-08 

99.962% 

Jul-08 

99.920% 

Aug-08 

99.967% 

Sep-08 

99.929% 

Oct-08 

99.972% 

Nov-08 

99.982% 

Dec-08 

99.955% 

Mass  Storage  Recovery  and  Expansion 

Disasters  may  strike,  but  the  DREN  is  ready.  With  large,  remote  mass  storage  poised  for  data  recovery,  customers 
can  feel  at  ease;  the  DREN  is  used  to  transfer  large,  single  stream  data  flows  in  the  multiple  Gigabit  range  to 
mass  storage,  continuously.  New  protocol  architectures  to  keep  up  with  technological  advances  use  9,000  bytes 
maximum  transmission  units.  Massive  data  transfers  occur  from  coast-to-coast  through  high-end  tuning  of  DREN's 
computational  resources. 

Internationally,  the  DREN  enables  key  exchange  points  in  Chicago,  Seattle,  Maryland,  and  California.  International 
science  exchanges  with  the  Australian  Meteorological  and  Oceanographic  Society  and  Asian  Pacific  Advanced 
Networks  projects  empower  DREN  personnel  to  keep  in  step  with  on-going  competitive  advances  around  the 
world. 


The  DREN  continues  to  expand 
as  new  computer  technology, 
mass  storage,  and  distributed 
computing  environments  place 
growing  demands  on  bandwidth 
and  network  architecture.  Core 
architecture  will  grow  to  40  gigabits 
per  second  or  more  over  the  next 
few  years  and  optics  architecture 
will  play  an  increasingly  central  role 
in  the  DREN  core. 


As  a  result  of  the  annual  survey, 
conducted  by  the  HPCMPO,  the 
DREN  received  high  customer 


5=  Strongly  Agree 
A-  Moderately  Agree 
3=  Neutral 

2=  Moderately  Disagree 
1  =  Strongly  Disagree 


Network  Availability  Files  Transfer  Rate  Allowable  File  Size  Overall  Bandwidth 
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satisfaction  scores  in  2008,  especially  in  network  availability  and  allowable  file  size.  The  DREN  received  a  score  of 
3.96  meeting  overall  customer  satisfaction. 

Internet  Protocol  Version  6  (IPv6) 

As  part  of  DoD's  information  age  transformation,  the  DREN  is  emerging  as  an  important  contributor  to  combat 
power  and  protection.  On  June  9,  2003,  the  DoD  CIO  issued  a  memorandum  stating  that  the  DoD  information 
infrastructure  of  the  future  would  depend  on  the  effective  implementation  of  IPv6  in  building  a  DoD-wide  Global 
Information  Grid  (GIG)  to  achieve  the  DoD's  Network-Centric  Operations  (NCO)  and  warfare  goals.  DREN  was 
the  first  DoD  IPv6  network  to  accomplish  the  following: 

1.  To  enable  the  entire  DREN  WAN  to  routinely  support  end-to-end  IPv6  traffic  between  and  among  all  DREN 
sites  and  multiple  external  peering  networks. 

2.  To  maintain  the  performance  and  security  levels  of  the  IPv6  pilot  at  the  levels  existing  on  the  IPv4-only  side  of 
the  network. 

3.  To  develop  a  process  that  facilitates  activation  of  the  IPv6  protocol  suite  at  all  DREN  sites  within  minutes. 

4.  To  enable  IPv6  in  selected  applications  provided  by  the  HPCMP. 

5.  To  document  and  provide  lessons  learned  to  help  facilitate  IPv6  implementations  at  all  levels  and  across  Federal 
Agencies  on  the  DREN  IPv6  knowledge  base  Web  site,  https://kb.v6.dren.net/  (Note:  This  is  a  CAC-enabled 
Web  site  that  will  be  opened  for  sharing  across  the  Federal  Agencies). 

The  DREN  achieved  most  IPv6  pilot  goals  by  FY  2008  with  less  than  $100,000  in  additional  funding  and  existing 
personnel.  The  DREN  team  took  little  time  to  enable  IPv6  at  the  IPv6  pilot  sites  and  shared  lessons  learned  with  many 
organizations  through  conference  presentations,  journal  papers,  magazine  articles,  interviews,  the  IPv6  knowledge 
base,  and  participation  in  DoD,  Intelligence  Community,  Federal,  and  industry  working  groups  and  workshops.  In 
2009,  the  DREN  will  complete  the  transition  in  implementation,  as  well  as  in  mindset,  from  an  IPv4  networks  that 
supports  IPv6  to  an  IPv6  network  with  legacy  support  for  IPv4.  The  "HPC  Tests  Network  Modernization  for  DoD" 
success  story  on  p.  38  discusses  the  IP  software  responsible  for  retrieving  data  from  one  location  and  sending  it  to 
another  location  on  a  network.  In  this  success  story,  John  Baird  discusses  IPv6  as  a  next  generation  protocol  that 
offers  a  wider  range  of  support  compared  to  IPv4. 

Security  and  Innovation 

As  advisors  on  DoD  control  boards  and  technical  councils,  the  DREN  contribution  to  the  DoD  networking  and  security 
communities  creates  resounding  effects.  The  DREN  fends  off  hostile  acts  of  intrusion  and  network  compromise  with 
their  Tier  2  DoD  Computer  Emergency  Response  Team.  In  FY  2008,  the  DREN  team  released  a  new  CSA  Tool  v5.2.1 
that  integrates  baseline  configuration  compliance  modules  and  support  issues  for  operations.  DREN  offers  all  of 
their  customers  Computer  Network  Defense  Services.  These  types  of  services  benefit  the  security  of  everyone  by 
introducing  a  wide  variety  of  toolsets  focused  on  network  monitoring,  incident  response,  and  vulnerability  assessment. 
Using  cryptographic  keys,  the  DREN  provides  Kerberos  network  authentication  system  as  a  tool  to  secure  valued 
information.  There  are  many  methodologies  in  place  for  the  DoD  and  other  Federal  Agencies  to  securly  exchange 
information  with  communities  of  interest  on  the  network. 

Powerful  resources,  such  as  the  VirtualToken  (VT)  project,  continue  to  provide  hardware-based  authentication  to 
remote  systems  for  research  and  collaboration.  As  partners  in  the  VT  project,  the  DREN  and  Ericsson  enhance 
ViPr  video  session  encryption  and  conferences  to  include  multicast  key  exchange,  multiple  video  and  audio  stream 
encryption,  and  key  control  for  conference  participants. 
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HPC  Tests  Network  Modernization  for  DoD 

The  DoD’s  HPC  Modernization  Program  Helps  Make  The  Transition  To 
IPv6 


by  John  Baird,  HPC  Modernization  Program 

The  Internet  is  nearly  forty  years  old,  and  it’s  starting  to 
show  its  age.  One  of  the  critical  pieces  that  needs  to  be 
modernized  is  the  Internet  Protocol  (IP)  software  that  runs 
the  network. 

IP  is  the  software  that  is  responsible  for  getting  data  from 
one  place  to  another  on  the  network.  Without  it,  a  computer 
could  only  talk  to  itself  -  no  email,  no  web  browsing,  and  no 
iTunes.  Most  of  the  Internet  today  runs  IP  version  4  (or  IPv4), 
which  was  designed  in  the  1960s  and  is  able  to  address 
about  4  billion  unique  devices  on  the  network.  When 
IPv4  was  first  deployed,  that  seemed  like  a  huge  number. 
But  by  the  early  1990s,  engineers  had  recognized  that  as 
networks  became  increasingly  complicated  and  as  more  of 
the  6+  billion  people  on  the  planet  owned  a  computer  or  a 
cell  phone  (or  both)  connected  to  the  Internet,  IPv4  would 
become  a  serious  impediment  to  progress.  By  1 996,  many 
of  the  standards  that  define  the  next  generation  protocol, 
known  as  IP  version  6  (or  IPv6),  were  complete. 

IPv6  is  different  in  several  important  ways  from  IPv4.  Two 
differences  stand  out.  First,  IPv6  uses  128  bits  for  device 
addresses  rather  than  just  32  bits  as  IPv4  does  which 
means  that  IPv6  can  directly  address  many  more  devices 
than  version  4  without  the  need  to  resort  to  complicated 
schemes.  Another  important  difference  is  that  IPv6  was 
designed  to  support  mobile  devices,  such  as  laptops  and 
cell  phones  and  mobile  users  who  want  to  remain 
connected  in  the  car  and  on  the  airplane  as  if 
they  are  at  home  or  in  the  office. 


In  2005,  the  Office  of  Management  and  Budget 
issued  a  memorandum  to  chief  information 
officers  of  Agencies  throughout  the  Executive 
branch  of  the  U.S.  government,  mandating  that 
network  backbones  support  IPv6  traffic  by  June 
2008.  Earlier,  the  Department  of  Defense  (DoD) 
had  gone  even  further,  identifying  the  deployment 
of  IPv6  networks  as  critical  to  achieving  its  net- 
centric  operations  and  warfare  (NCOW)  goals. 


The  DoD  information  infrastructure  of  the  future  will 
depend  on  the  effective  implementation  of  IPv6  to  build 
a  fully  distributed,  available,  and  secure  DoD-wide  Global 
Information  Grid  (GIG).  Enterprise-wide  deployment  of 
IPv6  was  initiated  by  the  Assistant  Secretary  of  Defense 
(Networks  &  Information  Integration  or  NII)/DoD  Chief 
Information  Officer  (ASD  (NII)/DoD  CIO)  in  June  2003.  That 
month,  the  ASD  (NII)/DoD  CIO  also  designated  the  Defense 
Research  and  Engineering  Network  (DREN)  operated  by  the 
DoD  High  Performance  Computing  Modernization  Program 
(HPCMP)  Office  as  the  first  DoD  IPv6  pilot  network,  in  part, 
because  DREN  is  DoD’s  recognized  network  for  the  DoD 
scientific  research,  engineering,  and  test  and  evaluation 
(T&E)  communities. 

Effective  implementation  of  IPv6  in  concert  with  other 
aspects  of  the  GIG  architecture  will  depend  on  the  successful 
outcome  of  a  full  spectrum  of  testing  methodologies, 
including  standards-based  analysis,  conformance, 
interoperability,  performance,  and  functional  testing.  To 
further  test  the  IPv6  functional  capabilities  of  multiple 
devices  operating  simultaneously  across  several  locations 
at  full  performance  levels,  which  are  required  to  achieve 
the  DoD’s  NCOW  goals,  all  of  these  testing  methodologies 
are  not  sufficient.  A  production-level  wide  area  network 
test  facility  is  required.  The  high-capacity,  low-latency, 
nationwide  DREN  is  such  an  IPv6  test  facility. 
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The  DREN  shares  lessons  learned  from  its  4+ 
years  of  operating  a  production  network  in  a  mixed 
IPv4/IPv6  environment  with  federal  Agencies 
and  Departments  outside  the  DoD  through  the 
IPv6  Working  Group  of  the  Federal  CIO  Council 
Architecture  and  Infrastructure  Committee. 

About  the  DoD  High  Performance 
Computing  Modernization  Program 


^  ^  End  of  the 
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Evolution  of  the  GIG 

When  the  FY  2007  DoD  IPv6  T&E  report  was  sent  to 
Congress  in  September  2007,  it  stated  that  one  of  the  best 
reasons  why  the  DREN  IPv6  pilot  has  been  developed  is  that 
it  has  given  the  DoD  community  a  production  environment 
to  more  directly  test  a  functional  network,  as  opposed  to 
a  closed,  limited  test  bed.  It  is  also  assisting  other  DoD 
agencies  in  configuration,  management,  security,  and 
deployment  of  an  IPv6  network.  The  lessons  learned  and 
research  to  be  conducted  for  this  effort  will  greatly  benefit 
the  DoD  community. 

Since  its  creation  in  1992,  DREN’s  major  thrusts  have  been 
high  performance  computing,  security,  and  testing  of  new 
technologies;  its  designation  as  a  DoD  IPv6  pilot  network 
early  in  the  DoD  IPv6  deployment  was  not  surprising.  The 
DoD  relies  upon  the  DREN  to  advance  basic  and  applied 
research,  enhance  engineering  design  and  development 
activities,  and  support  ongoing  test  and  evaluation  efforts, 
according  to  Mr.  Cray  Henry,  HPCMP  Director.  The  DREN 
has  been  and  will  continue  to  be  an  important  factor  in 
enabling  the  DoD  to  maintain  and  even  increase  the 
capabilities  of  the  Global  Information  Grid. 


The  DoD’s  High  Performance  Computing 
Modernization  Program  is  responsible  for  the 
acquisition  and  modernization  of  the  hardware, 
software,  networks,  and  expertise  that  provide 
some  of  the  world’s  most  advanced  computing 
capability  in  support  of  the  DoD  mission.  The  Program  has 
six  large  supercomputing  centers  and  supplies  expertise 
and  resources  at  numerous  other  military  sites  throughout 
the  country. 

For  more  information  on  the  IPv6  Pilot  Project,  go  to  http:// 
www.hpcmo.hpc.mil/Htdocs/DREN/dren_ipv6.pdf. 

Contact 

DoD  High  Performance  Computing  Modernization 
Program 

John  E.  West,  Senior  Fellow 
(601)  634-3629 

john.e.west@erdc.usace.army.mil 

IPv6  Pilot  Project 

John  Baird,  HPCMP  IPv6  Pilot  Implementation  Manager 

(703)  402-9638 

baird@hpcmo.hpc.mil 
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The  DREN  activated  HPCMP's  first  host-based  security  system  server  and  shipped  a  server  to  the  Secret  Defense 
Research  and  Engineering  Network  (SDREN)  NOC  for  installation.  Serving  customers  with  secure  and  personalized 
support,  the  DREN  sets  the  pace  for  enduring  progress  into  FY  2009  as  a  primary  computer  network  defense  security 
provider. 

DREN  Connectivity 

The  DREN  delivers  network  services,  such  as  IPv6,  multicast,  and  Voice  over  Internet  Protocol  (VoIP).  The  following 
map  displays  DREN  accessibility  and  the  benefits  satisfied  customers  enjoy  across  the  country. 


•  High  bandwidth-low 
latency  networks  with 
extremely  low  loss 

•  Classified  and  unclassified 
network  layers 

•  DREN  backbone  is 
OC-192 

•  Multicast-enabled  to 
support  video  broadcasts 

•  Dual-stacked  (IPv4  and 
IPv6) 
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Value  to  the  Customer 

DREN  customers  span  several  Services  and  Agencies  providing  network  support.  The  Army,  Navy,  Department  of 
Justice,  and  Joint  programs  need  the  DREN  to  accomplish  the  most  vital  tasks. 


Navy 

Provide  Computer  Network  Defense  Service  Provider  (CNDSP)  IT  services  to  ensure  efficiency  and  quality 
Space  and  Naval  Warfare  Systems  Command  (SPAWAR) 

Naval  Air  Systems  Command  (NAVAIR) 

Naval  Sea  Systems  Command  (NAVSEA)  Research  Development  Test  and  Engineering  (RDT&E)  sites 

Army 

•  Enhance  infrastructure  to  DREN  enclaves 

•  Implement  Internet  Protocol  version  6  (IPv6)  compliance 

•  Army  Training  and  Doctrine  Command  (TRADOC)  Battle  Labs 

•  Future  Combat  Systems  (FCS) 

•  Army  Test  and  Evaluation  Command  (ATEC)  Integrated  Range  Command  Center 

Air  Force 

Establish  RDT&E  security  boundaries  for  AF  enclaves 

Unmanned  air  vehicle  WAN  support  coast-to-coast  for  Global  Hawk  testing 

Joint  concept  technology,  “C2  Gap  Filler,”  demonstrations 

Air  Forces  Northern 

Air  Force  Operational  Test  and  Evaluation  Center  (AFOTEC) 

Joint  Surveillance  and  Target  Attack  Radar  System  (JSTARS) 

Airborne  High  Energy  Laser  Program 
Air  Force  Institute  of  Technology 

Joint  Programs 

Research  to  establish  a  distributed  test  environment  for  next  generation  Global  Command  and  Control  System  (GCCS) 
Provide  emergency  response  to  Detect  and  Respond  Services  for  the  Defense  Contract  Administration  Agency  (DCAA),  US 
Military  Academy,  and  US  Air  Force  Academy 

Other  Federal  Programs 

•  National  Aeronautics  and  Space  Administration  (NASA) 

•  United  States  Geological  Survey  (USGS) 

•  National  Oceanic  and  Atmospheric  Administration  (NOAA) 

•  Department  of  Justice  (DOJ) 

Federally  Funded  R&D  Centers 

Massachusetts  Institute  of  Technology  (MIT)  Lincoln  Laboratory  (LL) 

MITRE  Corporation 
Aerospace  Corporation 

Collaborative  Efforts 

Metropolitan  Research  and  Education  Network  (MREN) 

National  Science  Foundation  (NSF) 

Large  Scale  Network  (LSN) 

Joint  Engineering  Team  (JET) 

Hawaiian  Internet  Consortium  (HIC) 
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Software 
Applications 
Support  (SAS) 

Software  Applications  Support  (SAS)  provides  DoD  scientists  and  engineers  with  capabilities  to  design,  develop,  test, 
and  deploy  superior  weapons  systems  by  modeling  the  physical  world  to  support  the  Warfighter.  SAS  components 
include: 


Institutes 

Portfolios 

User  Productivity  Enhancement  and  Technology 
Transfer  (PET) 

Computational  Research  and  Engineering 
Acquisition  Tools  and  Environments  (CREATE) 

Software  Protection  Initiative  (SPI) 

Institutes 


Institutes 

Mission 

Expand  Service/ Agency  efforts  to 
provide  robust  production-level 
software  for  modeling,  simulation,  and 
computation  in  application  areas  of 
high  interest  to  DoD,  which  efficiently 
utilize  HPC  capabilities,  and  perform 
on  a  variety  of  HPC  platforms, 
with  focus  on  accuracy, 
efficiency,  reusability, 
and  scalability. 


Portfolio  Mission 

Provide  efficient,  scalable,  portable 
software  codes,  algorithms,  tools, 
models  and  simulations  that  run  on  a 
variety  of  HPC  platforms  and  are 
needed  by  a  large  number  of 
science  and  technology  (S&T) 
and  test  and  evaluation  (T&E) 
scientists  and 
engineers. 


Each  Institute  forms  a  critical  mass  of  experts  focused 
on  using  computational  science  and  high  performance 
computing  to  solve  the  DoD's  highest  priority 
challenges.  With  cross-Service  and  Agency  teams 
and  a  multi-disciplinary  approach,  Institutes 
transform  traditional  operational  processes  with 
computational  insight  by  using  legacy  or  newly- 
developed  computational  tools. 

Institutes  transform  DoD's  technological  challenges 
into  high  performance  solutions  through  the  following 
goals: 


CREATE 

Mission 

To  develop  and  deploy  three 
Computational  Engineering  tool  sets 
for  acquisition  program  engineers  to 
exploit  the  exponential  growth  in 
supercomputer  power.  These  tool  sets 
include  Aircraft  tools  (Aerodynamics  & 
Structures),  Ship  tools 
(Hydrodynamics  &  Structures)  and 
RF  Antenna  integration  tools 
(Electromagnetics). 


PET  Mission 

To  gather  and  deploy  the  best  ideas, 
algorithms  and  software  tools 
emerging  from  the  national  HPC 
infrastructure  into  the  DoD 
user  community. 


Enhance  existing  Service/ Agency  high  priority  projects. 

Utilize  science  and  technology  resources  to  develop,  apply,  and  transition  DoD  HPC 
software  applications. 


Build  HPC  experience  in  defense  laboratories,  engineering,  and  test  centers  for  defense  applications  among 
DoD  industrial  and  academic  scientists  and  engineers. 

Leverage  other  HPC  efforts  across  DoD,  government,  industry,  and  academia. 
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Awards  for  FY  2008 


The  DUSD(S&T)  evaluated  a  number  of  proposals  from  military 
Services  and  Agencies  in  FY  2007.  DoD  subject  matter  experts 
evaluated  and  approved  the  following  Institutes  to  begin  in  FY 
2008: 

Institute  for  High  Power  Microwave  (HPM)  Employment, 

Integration,  Optimization,  and  Effects,  AFRL,  Kirtland  AFB, 

NM. 

Mobile  Network  Modeling  Institute  (MNMI),  ARL,  Aberdeen 
Proving  Ground,  MD. 

Institute  for  Multi-Scale  Reactive  Modeling  and  Simulation  of 
Insensitive  Munitions  (MSRM-IM),  ARL,  Aberdeen  Proving 
Ground,  MD. 

Altogether,  the  HPCMP  supports  nine  Institutes,  three  new  and 
six  continuing  Institutes. 

Institute  for  High  Power  Microwave 
(HPM)  Employment,  Integration, 

Optimization,  and  Effects 

The  Institute's  vision  is  to  enable  seamless  integrated  end-to-end  simulation,  design  and  optimization  of  HPM 
weapons,  from  HPM  subsystem  to  platform  to  desired  effect.  It  links  multiple  software  development  efforts  into  a 
coherent  software  package  that  can  model  and  optimize  HPM  systems  on  platforms.  The  Institute  facilitates  the 
rapid  development,  acquisition,  and  fielding  of  effective  HPM  weapon  systems  providing  revolutionary  capabilities 
to  the  Warfighter. 

All  level  models  support  optimization,  experiment  design,  and  uncertainty  quantification  for  Analysis  of  Alternatives. 
This  framework  propagates  accurate  and  traceable  information  from  engineering  level  models  to  mission  and 
campaign-level  models  for  faster  acquisition  of  HPM  systems. 


Highly  integrated  end-to-end  HPM  system  model 


Engineering  Level  Models 

Abstract  Models 

- 1 

HPM  Source  Platform 

Propagation  Models  Pe  Models 

Pk  Models 

EMI/EMC 

Predictive  Fault  trees 

models  Advanced  network 
models 

EMC  -  Electromagnetic  Compatibility  EMI  -  Electromagnetic  Interference 
Pe  -  Predicting  Probability  of  Effect  Pk  -  Predicting  Probability  of  Kill 


Mission-Level  & 
Campaign-Level 
Analysis 
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The  Mobile  Network  Modeling  Institute  (MNMI) 

The  Institute's  vision  is  to  exploit  high  performance  computing  through  the  development  of  computational  tools 
that  enables  DoD  to  design  and  test  networks  at  sufficient  levels  of  fidelity  and  speed  to  understand  the  behaviors 
of  Net-Centric  Warfare  technologies  for  realistic  operational  conditions. 

MNMI  will  develop  multi-disciplinary  expertise  and  software  tools  that  transform  the  ways  in  which  DoD  model, 
simulate,  emulate,  and  experiment  with  dynamic  reconfigurable  mobile  warfighter  networks.  MNMI  conducts  three 
phases  in  an  HPC  environment:  experimentation,  simulation,  and  emulation. 


MANCT 


Experimentation 

Simulation 

Emulation 

•  Actual  hardware  in  field 

•  Objective  function 

•  PC  processors  represent 

experiment 

•  Behavioral  relationships 

nodes 

•  Traffic  generated  from 

•  Parameters 

•  Laboratory  environment 

applications 

•  Variables 

•  MANET  software  to  mode 

•  Realistic  scenarios 

node  movement  and  radio 

•  HPC  used  to  augment  and 

access 

stimulate  environment 

•  Actual  MANET  protocols 
run  on  nodes 

Institute  for  Multi-Scale  Reactive  Modeling  & 
Simulation  of  Insensitive  Munitions  (MSRM-IM) 

The  Institute  will  develop  a  science-based  capability  to  simulate  munition  response  to  insults 
by  capturing  the  effects  that  microstructural  heterogeneities,  inherent  in  composite  energetic 
materials  (EMs),  imposed  on  macroscopic  events.  This  effort  will  transform  the  modeling 
and  simulation  (M&S)  process  that  DoD  presently  utilizes  in  the  design  of  insensitive  munition 
(IM)-compliant  weapons  by  incorporating  essential  micro-  and  meso-level  modeling  capabilities 
needed  to  capture  key  physiochemical  properties  in  continuum  codes. 


msiji; 
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MSRM-IM's  vision  is  to  transform  costly  and  inefficient  DoD  IM  development  processes.  Results  include  safer 
munitions  and  quick  delivery  of  technology  at  lower  costs.  The  MSRM-IM  team  planned  a  strategy  in  FY  2008  to 
accomplish  five  strategic  goals: 

Produce  a  multi-scale,  physics-based  software  suite  that  describes  energetic  material  response  at  length  scales  ranging  from 
atomistic  through  mesoscale  to  continuum. 

Develop,  augment,  assess,  and  implement  HPC-based  software  for  mesoscale  simulation  of  chemical/physical  properties  of 
energetic  materials  and/or  materials  used  in  energetic  formulations  used  in  munitions. 

Augment  and  assess  existing  continuum  models  to  incorporate  the  results  and  methods  developed  from  atomistic  or 
mesoscale  simulations.  Apply  enhanced  continuum  models  to  salient  IM  practical  and  verification  problems. 

Develop,  augment,  assess,  and  implement  multiscale  homogenization  algorithms  using  consistent  or  rigorous  coarse  graining 
procedures  to  link  one  or  more  length  scales. 

Develop  and  demonstrate  a  robust  predictive  capability  for  system-context  response  of  munitions  to  IM  threats  through 
application  of  chemistry-  and  physics-based  algorithms  in  the  meso  and  continuum  regimes  of  energetic  materials. 


The  following  lists  six  continuing  Institutes: 

Continuing  FY  2008  Institutes 

Institute  for  Space  Situational  Awareness  (l-SSA) 

l-SSA  supported  Search  and  Determine  (SAD)  software  work  beginning  in  December.  SAD  is  a  track  association 
algorithm  developed  by  the  NRL  to  generate  candidate  orbits  from  uncorrelated  tracks.  The  astrodynamics  code 
constructs  satellite  orbits  from  uncorrelated  observations.  The  process  involves  constructing  orbits  from  pairs  of 
observations  that  are  not  associated  with  a  known  object.  This  resulted  in  a  greater  number  of  orbits  being  discovered 
than  was  previously  possible  and  the  run  times  have  been  significantly  reduced. 

Biotechnology  HPC  Software  Applications  Institute  for  Force  Health  Protection  (BHSAI) 

BHSAI  optimized  the  performance  of  the  original  Unified  Nucleic  Acid  Folding  (UNAfold)  source  program  by  simplifying 
the  I/O  operations  and  developing  a  Perl  interface  to  UNAfold.  UNAFold  is  a  comprehensive  software  package  for 
nucleic  acid  folding  and  hybridization  prediction. 

Institute  for  Maneuverability  and  Terrain  Physics  (IMTPS) 

The  IMTPS  focused  on  the  parallelization  of  new  models  to  support  the  countermine  test  bed,  implementation  of  a 
human  seismic  model,  and  the  numerical  formulation  of  simplex  elements  for  coupled  deformation  fluid  flow  problems. 

Battlespace  Environments  (BEI) 

BEI’s  development  activity  is  focused  on  getting  the  first  public  release  of  the  Earth  System  Modeling  Framework 
(ESMF)  3  series,  3.1  .Or.  In  this  release,  reworked  lower-level  general  data  structures  are  used  consistently  all  the  way 
up  through  the  higher  level  Field,  Bundle,  State,  and  Component  classes.  The  interfaces  at  the  Field  and  Bundle  level 
have  been  modified  to  correspond  to  the  new  Grid  and  Array  data  structures  on  which  they  are  newly  based. 

Institute  for  HPC  Applications  of  Air  Armaments  (IHAAA) 

The  IHAAA  team  renamed  the  technical  area  called  Fluid  Structure  Interaction  to  Aircraft  Mechanics  and  Dynamics 
because  the  main  emphasis  of  the  work  being  conducted  in  this  technical  area  remains  F-1 6  Limit  Cycle  Oscillations 
(LCO).  Joint  Model  Library  (JML)  software  2.1  was  released  9  June  2008;  beta  and  acceptance  testing  completed  as  of 
8  July  2008. 

HPC  Institute  for  Advanced  Rotorcraft  Modeling  and  Simulation  (HI-ARMS) 

HI-ARMS  focused  on  the  critical  path  elements  of  the  HI-ARMS  software  development  process.  The  Schools 
Interoperability  Framework  (SIF)  is  a  Python-based  framework  that  links  all  the  constituent  HI-ARMS  modules,  including: 

•  NSU3D  -  the  near-body  CFD  code 

•  HiMARC  -  the  off-body  adaptive  Cartesian  solver 

•  RCAS  -  the  structural  dynamics  and  comprehensive  analysis  module 

•  PUNDIT  -  the  domain  connectivity  module 
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Portfolios 


Nine  years  ago,  the  HPCMP  initiated  integrated  Portfolios  to  focus  on  multidisciplinary  challenges.  Portfolio 
development  teams  accelerate  technological  solutions  with  efficient,  scalable,  and  portable  software  codes, 
algorithms,  tools,  models,  and  simulations.  Teams  emphasize  reusability,  scalability,  and  portability.  This  initiative 
produces  a  new  generation  of  world-class  scientists  and  engineers  trained  in  scalable  software  techniques  to  reduce 
future  costs  and  increase  our  defense  capabilities. 

The  H  PCMP  provides  the  DoD  with  results  that  advance  the  design,  acquisition,  and  utilization  of  military  technologies 
and  aid  in  the  development  of  improved  military  capability  for  the  21 st  century.  In  FY  2008,  the  Rocket  Motor  and 
Warhead  Impact  Modeling  (RMWIM)  Portfolio,  formerly  known  as  Insensitive  Munitions  (IM),  provided  the  capability 
for  the  DoD  to  predict  the  severity  of  violence  from  explosions  and  costs  to  replace  military  assets.  With  F1PC  tools, 
the  RMWIM  Portfolio  empowers  DoD  decision  makers  to  thwart  undue  warfighter  injuries  or  deaths. 

Rocket  Motor  and  Warhead  Impact  Modeling  (RMWIM) 

The  RMWIM  Portfolio  enhances  legacy  modeling  and  simulation  tools  to  predict  the  response  of  munitions  to  bullet 
and  fragment  impact  (BFI).  To  this  end,  RMWIM  uses  Department  of  Energy  (DOE)  multi-physics  codes  to  model 
BFI  against  rocket  motors  and  confined  energetic  warheads.  This  Portfolio  addresses  the  issue  of  munition  sensitivity 
in  design,  storage,  packaging,  and  transport. 

The  end  product  is  to  provide  government  and 
industrial  munitions  designers  with  a  new  toolkit 
to  design  and  develop  munitions  compliant 
with  DoD  regulations  for  effectiveness,  safety, 
and  insensitivity  to  prescribed  insults.  The 
advancement  of  state-of-the-art  BFI-compliant 
munitions  systems  expands  the  fidelity  of  tools 
and,  ultimately,  provides  security  to  the  Warfighter 
on  the  ground. 

In  FY  2008,  the  RMWIM  Team  led  by  the 
Department  of  Army  at  Picatiinny  Arsenal, 
successfully  completed  implementation  of  an 
implicit  method  and  demonstrated  results  for  a  test  case.  Efforts  continue  to  improve  the  multiphase  flow  aspects  of 
the  modeling  and  include  enhancements  in  the  mesh  motion  terms;  the  variable  particle  velocities  for  the  multiphase 
flow;  and  the  convective  burn  model. 


POSSIBLE  REACTIONS  WHEN  FRAGMENT  IMPACTS  A 
CASED  ENERGETIC  MATERIAL 


PROMPT  SHOCK-TO-DETONATION 


PENETRATION  -  OVERVENTED 
BURN  /  EXPLOSION 


*1 

k 

& 

PENETRATION  -  OVERVENTED 
BURN  TO  EXPLOSION 


T.L.  Boggs  and  C.W.  Dickinson.  “Activities  of  the  Technical  Corporation  Program  W-Action  Group.”  1991 
JANNAF  Propulsion  Systems  Hazards  Subcommittee  Meeting,  CPIA  Publication 


User  Productivity  Enhancement  and  Technology 
Transfer  (PET) 

PET  enables  the  DoD  F1PC  user  community  to  make  the  best  use  of  F1PC  capacity  and  provides  and  extends  the 
range  of  DoD  technical  problems  solved  on  F1PC  systems.  PET  is  enhancing  the  total  capability  and  productivity  of 
the  FIPCMP's  user  community  through  HPC-related  science  and  technology  support,  training,  collaboration,  tool 
development,  support  for  software  development,  technology  tracking  and  transfer,  and  outreach  to  users. 
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Groundbreaking  PET  work  by  the  Mississippi  and  Ohio  State  (MOS)  University  Consortium  and  High  Performance 
Technologies,  Incorporated  (HPTi)  helps  the  HPCMP's  users  provide  tangible  results  for  the  Warfighters  of  today 
and  tomorrow.  The  following  success  stories  highlight  PET  achievements:  "Simulation  Support  for  the  MRAP  Vehicle 
Program",  "Large  Scale  Parallelization  of  TDDFT  in  GAMESS",  and  "Parallelization  and  Enhancement  of  RMA2 
(TABS-MD)." 

PET  teams  performed  computations  to  investigate  and  improve  Mine  Resistant  Ambush  Protected 
(MRAP)  armor  design  through  weapon-target  interactions  and  parametric  numerical  studies.  Analyses 
of  ballistic  behavior  of  advanced  ceramics  and  composite  materials  for  the  vehicle  ultimately  save  lives. 

These  PET  investigations  helped  counter  enemy  threats  through  computational  ingenuity.  See  page  49 
to  read  more  about  the  MRAP. 

"The  modeling ,  analysis ,  and  assessment  you  performed  were  precise  and  showed  the  excellence  of  your 
endeavor ."  -Todd  Bjerke,  Modeling  and  Simulation  (M&S)  MRAP  Coordinator. 

"Large  Scale  Parallelization  of  TDDFT  in  GAMESS,"  the  PET  team  discusses  the  GAMESS  code,  widely 
used  in  the  DoD  CCM  community  for  research,  and  the  HPCMP's  modifications  to  GAMESS.  See  page 
50  to  learn  more  about  the  GAMESS  performance  enhancement. 

"The  Parallelization  and  Enhancement  of  RMA2"  success  story  discusses  the  RMA2,  a  2-D  hydrodynamic 
model,  widely  used  for  the  modeling  of  rivers  and  watershed.  See  page  51  to  learn  more  about  the 
model. 

In  FY  2008,  PET  contracts  offered  47  training  events,  attended  by  579  students,  covering  subjects  ranging  from  code 
profiling  and  error  estimators  to  user  training  on  codes,  such  as  FLUENT,  ABAQUS®,  EnSight,  and  Xpatch®.  PET 
video  courses  instruct  students  online  throughout  the  country. 


Courses 

ACAD  Workshop 
Advanced  GridGen 
ANSYS 
Atomistix 

Case  Management  System  for  CFD  Solvers 
CFD/MIME/CFD  FSI 

Chemkin  Fundamentals  and  Applications  and  Design 
CTH 

Cybersecurity 

Data  Analysis  and  Assessment  Day  (EnSight) 
Fieldview  Workshop 
Fundamentals  of  XPatch 

FY90  Overview 

GPU  Programming 

HPC  Network  Simulation  for  the  DoD 

HPC  Network  Simulation  Workshop:  GTNetS 

Introduction  to  LS-DYNA 


Courses 

Introductory  MPI 

Joint  ENS/FMS  Workshop 

Large  Scale  Design  Optimization 

LS-DYNA  Applications  to  Protective  Structures,  Blasts, 
Vehicles  (IED  and  mines),  and  Home  Land  Security 

LS-PrePost  and  User  Defined  Materials  in  LS-DYNA 

OVERFLOW  Short  Course 

Parallel  Computing  in  CSM 

ParaView 

Performance  Evaluation  of  Multi-Language  Scientific 
Applications 

Photonics  Seminar 

Pointwise  and  Advanced  GridGen 

Seminar  on  Scalable  Chemistry  Code:  ACES,  Amber  9,  ATAT, 
CP2K,  LAMMPS,  NWChem,  ReaxFF  (GRASP) 

Shock  and  Detonation  Waves 

Smooth  Particle  Hydrodynamics  in  LS-DYNA 

SolidMesh 

STAR-CCM 

STARS3D 
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Simulation  Support  for  the  MRAP  Vehicle 

Program 


by  James  Cazamias,  CSM  On-site  -  ARL,  UAB 
Problem 

MRAP  vehicles  are  designed  to  survive  insurgent  and 
terrorist  attacks  and  ambushes,  which  cause  the  majority  of 
US  deaths  in  Iraq.  There  are  currently  designs  from  several 
vendors,  and  there  is  a  need  to  evaluate  the  competing 
designs.  There  is  also  a  continuing  need  to  improve  upon 
MRAP  designs  to  meet  more  sophisticated  weapons 
systems  employed  now  and  in  the  future  by  insurgent  and 
terrorist  forces. 

Methodology 

ARL  is  performing  computations  to  improve  the  armor  design 
for  the  MRAP  armored  vehicle  program.  In  support  of  this 
activity,  Cazamias  has  investigated  improved  methods  for 
simulation  of  weapon-target  interactions  and  for  parametric 
numerical  studies.  Efforts  have  focused  on  the  ballistic 
behavior  of  advanced  ceramics  and  composite  materials. 

Users  Supported 

This  work  supports  the  Armor  Mechanics  Branch  of  ARL- 
WMRD,  Scott  Schoenfeld  (Branch  Chief)  and  Todd  Bjerke 
(M&S  MRAP  Coordinator), 

DoD  Impact 

MRAP  is  currently  the  highest  priority  DoD  acquisition 
program.  This  effort  will  assist  the  Army  in  improving  armor 
for  these  vehicles  and  thereby  improve  survivability  of  the 
vehicle  and  safety  of  the  occupants. 


MRAP  vehicle  under  attack 


Bjerke  commented:  “Thank  you  very  much  for 
participating  in  the  M&S  portion  of  the  MRAP 
program.  This  was  a  critical  part  of  the  program 
and  your  persistence  in  developing  geometri¬ 
cally  accurate  and  physically-based  simulations 
paid  off.  ARL  management  was  impressed  with 
our  effort  and  I  commend  you  for  your  contribu¬ 
tion  to  a  successful  test  series.  The  modeling, 
analysis,  and  assessment  you  performed  were 
precise  and  showed  the  excellence  of  your  en¬ 
deavor.  Please  accept  my  gratitude  for  the  time 
and  obvious  effort  you  expended  on  improving 
our  Soldiers  survivability.  ” 
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Large  Scale  Parallelization  of  TDDFT  in 

GAM  ESS 


by  Michael  Lasinski  (CCM  Support,  ARL),  Nichols 
Romero  (CCM  Support,  ARL),  Shawn  Brown  (CCM 
Support,  PSC),  Anthony  Yau  (CCM  Support,  ARL), 
and  Jean  Blaudeau  (CPOC,  ARL) 

Problem 

PET  project  CCM-KY6-002  was  tasked  with  making  a  number  of 
performance  enhancements  to  the  GAMESS  quantum  chemistry 
code  (see  success  story,  “Large  Speedups  in  Quantum  Chemistry 
Code  -  Profiling  &  Optimization  of  GAMESS  on  MJM,”  above). 
These  include:  1)  integration  grid  replacement,  2)  parallelization 
of  the  time-dependent  density  functional  theory  (TDDFT),  and  3) 
profiling  and  optimization.  In  this  success  story,  we  focus  on 
the  parallelization  of  the  TDDFT,  and  present  benchmark  results 
demonstrating  its  scaling  on  DoD  HPC  systems. 

GAMESS  is  widely  used  in  the  DoD  CCM  community  for  research 
in  novel  propellants,  insensitive  munitions,  and  non-linear  optical 
(NLO)  materials.  Several  quantum  chemistry  methods  are 
available,  but  our  performance  enhancements  are  solely  focused 
on  the  (DFT)  and  (TDDFT)  methods.  DoD  users  perform  millions 
of  CPU  hours  per  year  of  calculations  with  GAMESS  on  DoD  HPC 
systems.  In  a  previous  PET  project,  CCM-KY5-002,  we  determined 
that  GAMESS  exhibited  superior  scaling  on  shared-memory  HPC 
architectures.  Hence,  our  benchmarks  focused  on  this  HPC 
architecture  type  with  our  target  computers  being  Eagle  (SGI  Altix 
3700  at  AFRL  MSRC),  Hawk  (SGI  Altix  4700  at  AFRL  MSRC),  and 
Sapphire  (Cray  XT3  at  ERDC  MSRC). 

Methodology 

Our  modifications  to  GAMESS  focused  on  the  parallelization 
of  the  TDDFT  method  which  initially  operated  only  serially. 
We  parallelized  both  the  TDDFT  energy  and  TDDFT  gradient 
calculations  in  GAMESS  using  the  existing  DDI  library.  By  taking 
advantage  of  the  Tuning  and  Analysis  Utilities  (TAU)  Performance 
SystemsTM,  we  were  also  able  to  identify  serial  bottlenecks  in 
the  GAMESS  TDDFT  implementation.  We  demonstrated  the 
performance  of  this  new  TDDFT  parallelization  capability  on  an 
input  file  for  Coumarin  152A,  a  dye  used  in  lasers.  Figure  4 
shows  the  parallel  speed-up  [extrapolated  from  a  normalization  to 
16  Processing  Elements  (PEs)]  for  a  TDDFT  gradient  calculation 
from  1 6-51 2  PEs  on  Sapphire.  We  have  also  run  the  same  input 
file  on  Hawk  for  256  and  500  processors  (not  shown).  Both  the 

v _ 


Number  of  Processing  Elements 

Parallel  speed-up  of  Coumarin  152A  TDDFT  gradient  calculation 
on  Sapphire  (normalized  by  extrapolation  from  16  processors) 

energy  portion  and  the  gradient  portion  of  the  TDDFT  perform 
above  50%  scaling  even  at  51 2  PEs.  The  TDDFT  components  of 
the  calculation  scale  better  than  the  DFT  counterparts  due  to  less 
computation  in  the  latter. 

Users  Supported 

The  DoD  users  supported  by  our  work  include  Jerry  Boatz  and 
Project  Partners  Douglas  Dudis,  Ruth  Pachter,  and  Todd  Yeates, 
as  well  as  all  HPCMP  users  of  GAMESS. 

DoD  Impact 

Performing  DFT  and  TDDFT  calculations  on  complex  molecules  is 
of  particular  interest  to  the  DoD  CCM  community,  and  significantly 
impacts  modeling  of  high-energy  density  materials  used  in 
propellants,  NLO  materials  used  for  sensor  applications,  and 
energy  storage  used  for  battery  simulations.  More  efficient 
execution  of  GAMESS  reduces  time-to-delivery  for  several  DoD 
material  research  and  development  efforts.  The  previous  version 
of  GAMESS  possessed  only  serial  implementations  of  both  the 
TDDFT  energy  and  gradient  calculations  which  limited  the  size 
of  molecules  that  could  be  treated  with  TDDFT.  As  the  graph 
shows,  a  500  processor  run  will  run  —400  times  as  fast  as  the 
serial  run  -  enabling  much  larger  molecules  to  be  simulated  with 
this  method,  greatly  expanding  the  capability  of  the  code.  This 
parallel  implementation  of  TDDFT,  along  with  a  number  of  other 
DFT  optimizations  and  a  newly  implemented  Lebedev  quadrature 
grid,  are  available  in  the  1 1APRIL2008R1  version  of  GAMESS. 
This  is  presently  the  default  version  on  Hawk,  Eagle,  Sapphire, 
and  Jade  ERDC  MSRC. 
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Parallelization  and  Enhancement  of  RMA2 

(TABS-MD) 


by  Jeff  Hensley,  EQM  On-site  -  ERDC,  UT 
Problem 

RMA2  is  a  2-D  depth-averaged  finite  element  hydrodynamic 
model  widely  used  for  surface  water  modeling.  However, 
its  usefulness  for  newer,  large  problems  has  been  restricted 
because  it  has  been  limited  mostly  to  shared  memory 
machines. 

Methodology 

Two  different  parallel  versions  of  RMA2  were  completed  using 
a  distributed  solver  (MUMPS).  Additionally,  unnecessary 
code  was  eliminated  and  other  improvements  made.  As 
a  result  of  these  improvements,  the  total  run  time  on  one 
large-scale  problem  was  reduced  by  nearly  40%. 

Users  Supported 

Joe  Letter  and  Barbara  Donnell  (ERDC/CHL) 

DoD  Impact 

RMA2  is  a  widely  used  tool  for  the  modeling  of  rivers  and 
watersheds,  and  is  a  critical  tool  for  the  analysis  of  flood 
control  and  environmental  impact  studies  in  many  USACE 
projects  across  the  United  States.  However,  the  need  for 
higher-resolution  modeling  has  increased  the  memory 
requirements  and  the  size  of  the  meshes  needed  to  a  point 
where  HPC  platforms  are  needed.  The  code  was  unable 
to  take  advantage  of  general,  distributed  memory  parallel 
machines.  The  effort  undertaken  by  Hensley  has  made  it 
possible  to  use  these  HPC  resources. 


Letter  commented:  “The  Corps  finite  element 
modeling  system  TABS-MD  supports  ERDC 
and  Corps  District  modelers  on  flood  control, 
navigation,  and  environmental  quality  studies. 
The  retirement  of  the  Digital  and  SGI  parallel 
machines  put  a  number  of  projects  in  jeopardy 
of  being  significantly  impacted.  Your  assistance 
in  porting  the  RMA2  code  to  the  XT3  has  kept 
at  least  three  projects  that  I  am  involved  in  on 
track.  This  could  not  have  been  accomplished 
without  your  knowledge  and  skill  with  the  HPC 
machines.  Thanks  for  your  help.  ’’ 
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Courses 

Courses 

Introductory/Intermediate  MATLAB 

Training  in  CCM  Scientific  Visualization 

Unstructured  Grid  Technology  for  CFD 

Surveying  Key  Issues  in  Shock  and  Detonation  Waves 

Using  LSPREPOST  and  dummies  in  LS-DYNA 

Webcasts 

CFD/MIME/CFD  FSI 

GPU  Programming 

ParaView  Developers  Course 

Pointwise  and  Advanced  GridGen 

While  the  Online  Knowledge  Center  (OKC)  contains  online  training  opportunities,  as  well  as  points-of-contact  for 
scientific  and  technical  support,  it  also  is  a  repository  of  PET-developed  technical  reports  and  presentations.  The 
OKC  front  page  features  computational  electromagnetics  and  acoustics  for  all  PET  functional  areas.  Complete 
information  regarding  PET  activities,  training,  current  and  past  projects,  and  technical  reports  can  be  found  at 
https://okc.erdc.hpc.mil. 

HPCMP  technical  and  program  management  emphasize  and  encourage  our  entire  team  of  functional  experts,  on-site 
personnel,  principal  investigators,  and  business  administrators  to  focus  on  the  key  goals  of  PET  program:  technology 
transfer,  user  productivity,  and  DoD  mission  impact. 

This  map  displays  the  teams,  comprised  of  experts,  from  a  range  of  universities  and  companies,  that  provide  one- 
on-one  scientific  assistance  to  HPCMP  users. 


DeposiTech,  Inc.,  Haslet,  TX 
STAR  Institute,  Boulder,  CO 

Computer  Sciences  Corporation,  Huntsville,  AL 
High  Performance  Technologies,  Inc.,  Reston,  VA 
Georgia  Institute  of  Technology,  Atlanta,  GA 
Mississippi  State  University,  Starkville,  MS 
Brown  Deer  Technology,  Bel  Air,  MD 
University  of  New  Mexico,  Albuquerque,  NM 
Ohio  Supercomputer  Center/The  Ohio  State 
University,  Columbus,  OH 

Pittsburgh  Supercomputing  Center/Carnegie-Mellon 
University,  Pittsburgh,  PA 
Science  Applications  International  Corporation, 
Orlando,  FL 

University  of  Hawaii*,  Honolulu,  HI 
Arizona  State  University,  Tempe,  AZ 


14.  University  of  Tennessee,  Knoxville,  TN 

15.  University  of  Texas,  Austin,  TX 

16.  Stanford  University,  Palo  Alto,  CA 

17.  University  of  Alabama,  Birmingham,  AL 

18.  University  of  Minnesota,  Minneapolis,  MN 

19.  Pensylvania  State  University,  University  Park,  PA 

20.  University  of  Texas*,  El  Paso,  TX 

21.  SFA,  Inc.,  Crofton,  MD 

22.  University  of  Illinois,  Urbana-Champaign,  IL 

23.  Paratools,  Eugene,  OR 

24.  ACES  Q.C.,  Gainesville,  FL 

25.  Metron,  Inc.,  Reston,  VA 

26.  University  of  California  at  San  Diego 

27.  Florida  International  University*,  Miami,  FL 

28.  Jackson  State  University*,  Jackson,  MS 

*  denotes  Minority  Serving  Institution  (MSI) 


Legend 

^  HPTi  Team 

MOS  University  Consortium 
W  HPTi  and  MOS 
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Computational  Research  and  Engineering  Acquisition 
Tools  and  Environments  (CREATE) 

CREATE  became  a  funded  program  in  FY  2008,  chartered  to  develop  and  deploy  three  sets  of  physics-based 
computational  engineering  design  tools  for  acquisition  programs: 

CREATE-Air  Vehicles  (AV)  -  Military  Air  Vehicle 
Design 

CREATE-Ships  -  Military  Ship  Design 

CREATE-Radio  Frequency  (RF)  Antennas  -  Antenna 
Design  and  Integration  with  Platforms 

In  FY  2008,  a  fourth  project,  CREATE-Meshing  and 
Geometry  (MG),  was  added  to  address  the  meshing  and  geometry  needs  of  the  other  three  projects. 

CREATE  tools  will  enhance  the  productivity  of  the  DoD  acquisition  engineering  workforce  by  providing  high  fidelity 
design  and  analysis  tools  with  capabilities  greater  than  today's  tools,  reducing  the  acquisition  development  and  test 
process  cycle.  CREATE  projects  will  provide  enhanced  engineering  design  tools  within  three  to  four  years  after 
project  start  dates  and  mature  tools  at  the  end  of  the  12-year  project  schedule.  Final  tool  set  design  will  capitalize 
on  the  next  generation  of  supercomputers  available  in  2020. 

In  FY  2008,  the  HPCMP  established  four  CREATE  Boards  of  Directors  (BoDs):  CREATE-AV,  CREATE-Ships,  CREATE- 
RF,  and  CREATE-MG.  During  FY  2008,  the  CREATE  BoDs  met  to  endorse  requirements  and  software  gaps  cited  in 
the  Initial  Capabilities  Documents. 


CREATE 

Computational  Research  and  Engineering  Acquisition  Tools  and  Environments 


No  single  DoD  institution  has  the  staff  with  the  broad  collection  of  skills  needed  to  execute  all  elements  of  one 
project;  therefore,  each  CREATE  project  is  multi-institutional,  with  participation  from  Air  Force,  Navy,  and  Army 
engineering  centers,  laboratories  and  acquisition  organizations,  DoD  contractors,  and  other  federal  agencies. 


CREATE-AV 


Helios 


CREATE-AV  provides  significant  strategic  and  competitive  advantages  for 
the  DoD  aeronautical  weapon  systems  acquisition  workforce  through 
Computationally  Based  Engineering  (CBE)  and  next  generation 
computers.  CBE  software  products  have  a  three-fold  impact: 
increased  capability  of  acquisition  program  engineers; 
reduced  workload  through  streamlined  and  more  efficient 
acquisition  workflows;  and  minimized  need  for  rework 
due  to  early  detection  of  design  faults  and  performance 
anomalies. 


The  CREATE-AV  Project  is  headquartered  at  NAVAIR,  Patuxent 

River,  MD,  but  includes  elements  that  provide  project  oversight,  project  management, 
requirements  gathering,  CBE  software  product  development,  product  verification,  validation 
and  quality  control,  and  technology  transition. 


Kestrel 


Shadow-Ops 
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The  FY  2008  initial  CREATE-AV  BoD  consisted  of  the  following  senior  representatives  from  the  service  engineering 
organizations:  Dr.  Ed  Kraft  (Chair),  Chief  Technologist,  Arnold  Engineering  Development  Center  (AEDC),  Air  Force 
Materiel  Command;  Dr.  William  Lewis,  Director,  Aviation  Engineering,  US  Army  Aviation  and  Missile  Research 
(AMRDEC),  Development  &  Engineering  Center;  Dr.  Michael  Scully,  Senior  Staff  Engineer,  US  Army  Aero-flight 
Dynamics  (AFDD)  Directorate,  AMRDEC;  Mr.  Richard  Gilpin,  Director,  Aircraft  &  Unmanned  Aerial  Vehicles, 
NAVAIR;  and  Mr.  Jorge  Gonzalez,  Technical  Director,  Engineering,  AFMC  Headquarters  (HQ  AFMC/EN);  and  Mr. 
Cray  Henry,  Director,  DoD  HPCMP. 

CREATE-AV  Teams  began  collaboration  with  researchers  and  project  planning  in  FY  2008  to  fulfill  the  needs  of  the 
defense  acquisition  workforce. 

Planning 

The  Planning  Team,  comprised  of  senior  acquisition  engineers  across  the  Services,  ensured  that  project  activities  and 
products  responded  to  the  needs  of  the  defense  acquisition  engineering  workforce  by  providing  project  management 
and  developers  with  cognizance  of  key  acquisition  process  workflows  and  code  building  environment  impact, 
developing  plans  to  build  bridges  between  developers  and  the  targeted  workforces,  and  providing  recommendations 
to  the  Board  of  Directors. 

Kestrel  Software  Product  Development 

The  Kestrel  Team  was  established  in  FY  2008  and  core  team  members  are  hosted  at  the  Air  Force  SEEK  EAGLE  Office 
(AFSEO)  at  Eglin  AFB,  FL  and  University  of  Alabama,  Birmingham,  AL.  Kestrel  will  enable  integrated,  CBE-based, 
test  and  analysis  of  fixed-wing  aircraft  designs  via  simulation  methods  to  verify  vehicle  performance;  perform  flight 
certifications  and  qualifications;  rehearse  ground-based  and  full-scale  flight  tests;  and  evaluate  planned  or  potential 
operational  use  scenarios.  Through  regular  release  cycles,  the  team  will  deliver  the  full  suite  of  Kestrel  capabilities 
over  several  years.  The  first  release  of  Kestrel  to  targeted  acquisition  engineering  organizations  is  scheduled  for 
late  FY  2009. 

Helios  Software  Product  Development 

The  Helios  Team  is  hosted  at  AMRDEC,  AFDD  at  Moffett  Field,  CA,  with  other  key  members  at  the  University  of 
Wyoming.  During  FY  2008,  Helios  organized  a  Technical  Advisory  Board  comprised  of  technical  experts  from 
defense  laboratories,  NASA,  the  domestic  rotorcraft  industry,  and  academia,  to  ensure  the  physical  accuracy, 
computational  efficiency,  and  usability  of  the  Helios  product  within  the  defense  acquisition  engineering  workforce. 
Through  regular  release  cycles,  the  team  will  deliver  the  full  suite  of  Kestrel  capabilities  over  several  years.  The  first 
release  of  Helios  is  scheduled  for  late  FY  2009. 

Helios  will  ultimately  enable  integrated,  CBE-based,  test  and  analysis  of  rotary-wing  aircraft  designs  via  simulation 
methods  to  verify  vehicle  performance;  perform  flight  certifications  and  qualifications;  rehearse  ground-based  and 
full-scale  flight  tests;  and  evaluate  planned  or  potential  operational  use  scenarios.  In  time,  Helios  will  also  allow 
variable  fidelity  physics  selection  to  facilitate  effective  CBE  use  in  early-phase  acquisition  process  workflows. 

Shadow-Operations 

The  Shadow-Operations  Team  is  headquartered  at  NAVAIR  in  Patuxent  River,  MD.  In  collaboration  with  management 
of  targeted  acquisition  programs,  this  team  "shadows",  or  copies,  current  acquisition  activities  using  the  newest  releases 
of  CREATE-AV  CBE  software  products  to  positively  impact  the  targeted  acquisition  program,  gather  requirements, 
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and  accumulate  real  acquisition  test  cases.  The  targeted  acquisition  program  archives  case  conditions,  simulation 
results,  and  test  data  for  validation  testing  of  future  releases  of  CREATE-AV  CBE  software  products.  Activities  of  the 
Shadow-Operations  Team  support  all  strategic  goals  of  the  project. 

In  FY  2008,  the  team  shadowed  the  following  acquisition  programs: 

EP-3E  Flight  Qualities  Assessment 
UCAS  Conceptual  Design 

JSF  Short  Take-off  and  Vertical  Landing  (STOVL)  Weapons  Bay  Emergency  Jettison 
JSF  Aero-Propulsion  Integration 
GBU-39  Weapon  Integration 

The  team  made  huge  impacts  on  the  EP-3E  assessment  and  UCAS  projects  this  year.  Through  the  use  of  computational 
fluid  dynamics  tools  to  analyze  different  flight  conditions,  the  Shadow-Operations  Team  reduced  the  required  time 
for  the  Navy  EP-3  reconnaissance  aircraft  Flying  Qualities  Flight  Clearance  from  one  year  to  three  months!  The 
elimination  of  subsequent  tests,  construction  costs,  and  flying  quality  reports  allows  the  acquisition  program  to  meet 
present  deployment  dates  in  the  forward  theater.  The  Navy  EP-3  combats  terrorism,  supporting  the  Warfighter  in 
today's  wars. 

Computational  fluid  dynamics  tools  assisted  the  team  for  the  optimization  study  of  the  proposed  UCAS  conceptual 
design.  The  Shadow-Operations  Team  identified  improvements  for  stability,  control,  and  potential  ratio  improvements 
of  lift  to  drag  from  50  percent  to  90  percent!  These  successes  not  only  eliminate  extra  costs,  they  provide  the 
Warfighter  with  state-of-the-art  tools  to  defend  our  nation. 


CREATE-RF  Antennas 


As  integral  components  of  DoD  weapon  systems,  antennas  perform  critical  communication,  identification,  and 
navigation  functions  to  increase  the  Warfighter's  effectiveness  and  survivability.  Antenna  systems  and  computational 

electromagnetics  (CEM)  tools  require  advanced,  fast,  and  accurate  methods 
to  take  advantage  of  upcoming  multi-node,  multi-core  high  performance 
computing  hardware  with  ease-of-use  to  encourage  more  users, 
eliminate  errors,  and  facilitate  productive  analysis. 


CREATE, 

CREATE-RF's  vision  is  to  develop  and  transition 
maintainable,  extensible,  validated,  and  productive 
geometry,  CEM,  and  user  interface  tools  to  support 
rapid  antenna  system  design  and  integration  into  DoD 
structures,  air,  sea,  land,  and  space-based  platforms  for 
an  estimated  20-year  life  span.  Properly  constructed  tools 
have  the  potential  to  increase  the  current  user  base  and  set  up 
antenna  systems  with  high  performance  at  low  cost.  Accuracy  and  advanced  computational  ability  to  obtain  systems 
that  meet  different  specifications,  eliminating  costly  redesigns,  saves  the  DoD  money,  time,  and  effort. 


In  addition,  CREATE-RF  aims  to  improve  the  ability  of  DoD  institutions  to  develop  and  exploit  large-scale  computational 
science  and  engineering  tools  by  building  the  technical  capability  of  DoD  institutions.  The  development  of  antenna 
modeling  capabilities  represents  the  best  tools  for  our  nation.  Therefore,  the  project  plans  to  employ  both  software 
and  hardware-based  protection  measures  to  safeguard  its  intellectual  property  during  the  development,  distribution, 
and  application  stages. 
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The  CREATE-RF  project  office  established  a  geographically  dispersed  team  working  on  the  same  codes  through  a 
collaborative  development  enclave.  The  project  headquarters  is  located  in  the  Sensors  Directorate,  AFRL,  Wright- 
Patterson  AFB,  OH,  and  several  participating  institutions  via  small  workgroups.  The  CREATE-RF  team  has  members 
from  the  Army,  Navy,  and  Air  Force. 

The  CREATE-RF  project  office  also  established  a  contracting  framework  to  obtain  required  software  engineering 
and  coding  support  by  industrial  contractors.  60  people  attended  an  'Industry  Day"  on  18  April  2008  at  Wright- 
Patterson  Air  Force  Base;  44  attendees  represented  27  different  companies  and  universities.  The  CREATE-RF  team 
also  hosted  various  software  training  classes  at  Wright-Patterson  AFB  for  the  Automated  Computer  Aided  Design 
tool  (ACAD);  ILANS,  a  Virtual  Electromagnetic  Design  (VED)-developed  graphical  user  interface;  and  CARLOS,  a 
VED-developed  CEM  code  engine. 

The  CREATE-RF  BoD  consists  of  the  following  members:  Dr.  Frederick  Tokarz  (Chair),  Technical  Advisor  for  Low 
Observables,  AFMC,  Aeronautical  Systems  Center;  Dr.  William  Baker,  Chief  Scientist,  Directed  Energy  Directorate, 
AFMC,  AFRL;  Dr.  Janet  Fender,  Chief  Scientist,  Air  Combat  Command;  Mr.  Dennis  DeCarlo,  Chief  Antenna  Systems 
Engineer,  NAVAIR;  Ms.  Betsy  DeLong,  Deputy,  Transition  and  Innovative  Naval  Prototypes,  Office  of  Naval  Research 
(ONR);  Dr.  Mahbub  Hoque,  Chief  Scientist,  Space  and  Terrestrial  Communication  Directorate,  Army  Communication 
Engineering  Research  and  Development  Electronic  Center;  and  Mr.  Cray  Henry,  Director,  DoD  HPCMP. 


Future  Plans 

CREATE-RF  foresees  FY  2009  as  the  first  execution  year  for  software  design,  integration,  and  test  activities.  More 
formal  oversight  and  tracking  tools  are  to  be  put  in  place  to  ensure  effective  program  management.  In  addition 
to  releasing  the  first  iteration  of  its  developed  software  tools,  CREATE-RF  and  its  partners  will  vet  the  requirements 
identified  in  its  Initial  Capabilities  Document.  CREATE-RF  expects  additional  focus  areas  to  emerge  in  FY  2009  based 
on  continuing  discussions  with  antenna  system  designers  and  their  organizations. 


CREATE-Ships 


Shock/Damage 


SHIPS 


New  ship  design  depends  on  new  technologies  and  concepts. 

While  existing  physics-based  computational  engineering  tools 
provide  a  base  for  research,  they  need  improved  accuracy  and 
more  complex  physics.  In  addition,  they  provide  design 
information  too  slowly  to  impact  the  system  design 
process  and  meet  time  constraints. 

To  remove  this  deficiency,  the  CREATE  physics- 
based  tools  will  use  the  next  generation  of  high 
performance  computers  to  provide  accurate 
design  information  within  timeframes  mandated 
by  the  Navy's  design  decision  process.  The  Ship 
Design  Project,  also  known  as  CREATE-Ships,  plans 

to  develop  the  engineering  software  to  support  a  reconfigurable  ship  design  and  acquisition  process.  HPC  engineering 
tools  will  create  an  optimized  total  warship  design  through  properly  designed  hull,  mechanical,  and  electrical 
systems  integrated  with  combat  and  other  mission  systems  early  in  the  acquisition  process.  Navy  development 
of  cost-effective  ship  designs  to  meet  performance  requirements  on  schedule  and  within  budget  is  feasible  with 
CREATE-Ships  solutions. 


Rapid  Design  &  Integration 
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The  Naval  Surface  Warfare  Center,  Carderock  Division  (NSWCCD),  West  Bethesda,  MD,  is  the  headquarters  for 
CREATE-Ships. 

CREATE-Ships  is  currently  composed  of  three  product  areas: 

Rapid  Design  and  Integration:  Comprehensive,  rapid  exploration  of  the  feasible  design  space 

Hydrodynamics:  Integrated  Hydrodynamics  Design  Environment  (IHDE)  and  high-fidelity  code  with  object- 
oriented  open  source  software 

Shock/Damage:  Enhanced  Dynamic  Systems  Mechanics -Advanced  Simulation  (DYSMAS)  for  close-in  underwater 
explosions 

Rapid  Design  and  Integration 

CREATE-Ships  breaks  new  ground  for  the  Navy's  LEAPS  product  model  database  to  accept  parallel  queries  for  ship 
design  data.  FY  2008  set  the  groundwork  for  many  activities  CREATE-Ships  will  execute  in  FY  2009  ,  including  multi¬ 
disciplinary  optimization,  using  response  surface  methodologies  to  replace  the  traditional  spiral  approach. 

Hydrodynamics 

CREATE-Ships  began  development  of  the  IHDE  for  use  by  naval  architects  and  hydrodynamicists  to  promote  design 
processes  and  provide  access  to  a  suite  of  analysis  software,  automated  meshing,  shape  optimization,  job  submission, 
and  validation  against  existing  experimental  data.  FY  2009  marks  the  IHDE's  first  formal  release. 

Planning  new  high-fidelity  code  for  the  Navy  using  available  object-oriented  open  source  software  began  in  FY  2008. 
CREATE-Ships  coordinates  this  effort  with  NSWCCD,  Pennsylvania  State  University  Applied  Physics  Laboratory, 
University  of  Iowa,  and  Science  Applications  International  Corporation  (SAIC). 

Shock/Damage 

The  DYSMAS  fluid-structure  interaction  software  creates  predictions  for  full  ship  shock  trials,  to  alleviate  potential 
environmental  impacts  and  demonstrate  ship  survivability  capabilities.  CREATE-Ships  and  other  members  of  the 
Shock/Damage  Product  Team,  NSWCCD,  Naval  Surface  Warfare  Center  Indian  Head  (NSWCIH),  and  Sandia 
National  Laboratory,  developed  detailed  enhancement  plans  for  FY  2008-2013. 

In  response  to  a  FY  2007  JASON  Defense  Advisory  Group  review  of  DYSMAS,  the  Shock/Damage  Product  Team's 
development  activity  initiated  required  structural  improvements  by  making  use  of  portions  of  Sandia's  Sierra  mechanics 
software  suite,  as  well  as  parallelization  of  the  fluid-structure  coupler  software.  CREATE-Ships  coordinated  efforts 
with  its  Technical  Advisory  Group  to  further  enhance  ship  design  methods.  These  enhancements  provide  both 
functionality  and  timeliness  improvements  to  support  ship  design  decisions. 

CREATE-Ships  Shock/Damage  Technical  Advisory  Group  includes  representatives  from  the  following  groups: 

Navy  Technical  Authority  for  Ship  Vulnerability 
Program  Executive  Office  (PEO)  Ships 
PEO  Aircraft  Carriers 
PEO  Submarines 
OSD  Live  Fire  T&E 
•  ONR 
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The  CREATE-Ships  BoD  consists  of  the  following  members:  RDML  T.  Eccles  (Chair),  Chief  Engineer,  Naval  Systems 
Engineering  Directorate,  NAVSEA;  Dr.  Walter  Jones,  Executive  Director,  ONR,  Mr.  Charles  Reeves,  Technical  Director, 
NSWCCD;  Mr.  Glen  Sturtevant,  Director  for  Science  and  Technology,  Program  Executive  Office  (SHIPS);  and  Mr. 
Cray  Henry,  Director,  DoD  HPCMP. 

With  CREATE-Ships  as  catalysts,  NAVSEA  initiated  the  development  of  a  comprehensive  Ship  Design  Tools  Roadmap 
as  a  plan  to  develop  ship  design,  analysis,  and  integration  computer  tools.  CREATE-Ships  and  its  Board  of  Directors 
approved  the  project's  requirements  and  initial  development  plans  for  shock/damage  and  hydrodynamics  performance 
analysis. 

CREATE-Ships  met  with  representatives  of  the  National  Shipbuilding  Research  Program,  from  General  Dynamics- 
Electric  Boat  and  Northrop-Grumman  Ship  Systems,  to  explore  mutual  interests  in  ship  design  and  analysis  tools  for 
ship  geometry  generation  and  exchange,  mesh  generation,  and  tool-to-tool  exchange  of  analysis  results. 

The  ONR,  NAVSEA,  and  CREATE-Ships  organized  one  of  six  workshops  on  the  ship  design  process  on  28-30  May 
2008.  Approximately  100  participants  from  Navy,  industry,  academia,  and  national  laboratories  attended  the  first 
workshop. 

Future  Plans 

CREATE-Ships  anticipates  FY  2009  as  an  execution  year  for  rapid  design  and  integration  activities,  as  well  as  a  HPC 
planning  year  for  additional  products.  CREATE-Ships  and  the  design  community  will  vet  the  requirements  for  the 
IHDE  to  continue  software  development  for  the  formal  release  in  FY  2009.  FY  2009  marks  the  final  decision  for 
the  high-end  software  development  also.  CREATE-Ships  expects  additional  product  areas  to  emerge  in  FY  2009, 
based  on  continuing  discussions  within  the  Navy  ship  design  community. 

CREATE-MG 

In  mid-FY  2008,  CREATE  added  the  Meshing  and  Geometry  (MG)  Project,  to  support  CREATE-AV,  CREATE-RF, 
CREATE-Ships,  and  their  stakeholders.  MG  provides  meshing  and  geometry  tools  for  automation,  interoperability, 
and  ease-of-use  for  CBE  software  for  each  major  CREATE  project. 

The  core  team  is  located  at  the  NRL  and  acts  as  a  joint  DoD  NRL/DOE  partnership  effort  to  leverage  existing 
geometry  and  meshing  technology  from  Sandia  National  Laboratory. 

The  FY  2008  CREATE-MG  BoD  consisted  of:  Dr.  Robert  Meakin  (Chair),  CREATE-AV  Project  Lead;  Mr.  Cray  Henry, 
Director,  HPCMP;  Dr.  Kueichien  Hill,  CREATE-RF  Project  Lead;  Mr.  Myles  Hurwitz,  CREATE-SHIPS  Project  Lead;  Dr. 
David  Fisher,  CREATE  Chief  Scientist;  and  Dr.  Steven  Owen,  Sandia  National  Laboratories/DOE. 

During  FY  2008,  the  project  efforts  focused  on  requirements  gathering,  formalization,  and  reconciliation,  and  project 
planning,  team-building,  and  staffing. 
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Outreach 

The  objective  of  HPCMP  Outreach  is  to  offer  opportunities  for  scientists  and  engineers  to  present  ideas  to  peers, 
young  people,  and  the  DoD  community.  Outreach  promotes  collaborative  relationships  among  the  DoD,  national 
science  communities,  and  Minority  Serving  Institutions  (MSIs),  and  encourages  active  involvement  to  achieve 
Program  goals.  Collectively,  all  HPCMP  departments  and  the  user  community  determine  outreach  success  by  actively 
participating  in  HPCMP  Outreach  efforts. 


In  2008,  HPCMP  Outreach  efforts  included: 

Users  Group  Conference  (UGC) 

DREN  Network  and  Security 
Conference 

Joint  Educational  Opportunities 
for  Minorities  (JEOM)  summer 
colloquium,  institutes,  workshops,  and 
scholarships  for  science,  technology, 
engineering,  and  mathematics  (STEM) 
faculty  at  Historically  Black  Colleges 
and  Universities  (HBCUs)  and  MSIs 

National  Defense  Science  and 
Engineering  (NDSEG)  Fellowship 
Program 

DoD  Institutes  for  Higher  Learning 
Program  -  Cadets  and  Midshipmen 
Summer  Projects 

Users  Group  Conference 
(UGC) 


Outreach 
Mission 

To  promote  the  HPCMP  and  foster 
the  exchange  of  ideas  among  HPC 
scientific  and  computational 
communities  within  the 
Department  and  at  the  national 
level. 


The  2008  DoD  HPCMP  UGC  was  held  on  July  14-17  in 
Seattle,  WA.  This  annual  conference  enables  the  DoD  S&T 
and  T&E  communities  to  share  their  computational  science  and 
engineering  results  with  colleagues  and  other  members  of  the 
community.  The  conference  brings  together  a  community  of  scientists 


Increase  the 
nvolvement  of 
underrepresented  groups 
in  the  areas  of 
computational  sciences  and 
HPC  through  outreach, 
mentorship,  and 
educational 
opportunities 


Increase  the 
technical 
community's 
awareness  of  program 
resources,  progress  and 
accomplishments 


Raise  the 
visibility  of  HPCMP 
services,  computing 
resources,  programs, 
and  research  done  at 
HPC  Centers 
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and  engineers  with  common  interests,  encourages  the  sharing  of  techniques  and  methods,  and  allows 
the  users  to  interact  with  staff  from  HPC  Centers  and  the  HPCMPO.  Users  gain  an  appreciation 
of  the  technical  breadth  of  the  entire  community.  In  FY  2008,  the  UGC  was  organized  by  the 
Program's  User  Advocacy  Group  and  the  F-IPCMPO  staff. 


The  UGC  included  six  invited  speakers:  Capt.  Vic  Addison  discussed  the  role  of  supercomputers 
for  the  Navy;  Dr.  Steve  Scott  summarized  the  advances  in  \~\PC  productivity  through 
CRAY'S  achievements  with  DARPA  High  Productivity  Computing  System  project;  Mr. 
Kevin  Gildea  summarized  the  advances  in  F1PC  productivity  through  IBM's  achievements; 
VADM  Daniel  Oliver  discussed  the  role  of  supercomputing  at  the  Naval  Postgraduate 
School;  Mr.  Doug  Ball  summarized  the  role  that  supercomputing  played  in  commercial  aircraft 
design  for  Boeing;  Dr.  Dan  Reed  discussed  emerging  fields  that  supercomputing  will  enable;  Mr.  Chris 
Castello  discussed  the  Google  Earth  project;  and  Dr.  David  Bacon  summarized  the  status  of  quantum  computing. 
Mr.  Cray  F-lenry  presented  the  Director's  Report  that  highlighted  new,  available  computing  power  to  the  DoD;  the 
start  of  three  new  Institutes  (insensitive  munitions,  high  power  microwaves,  and  network  modeling);  the  plans  for 
the  next  PET  contract;  the  status  of  the  CREATE  program;  and  the  Program  Office's  successful  move  from  Ballston 
to  Lorton,  VA. 


There  were  four  invited  presentations  on  technical  results  by  DoD  F-IPCMP  users:  Dr.  James 
Doyle  (NRL-MRY)  summarized  progress  on  predicting  weather  for  the  naval  battlespace 
environment  and  the  impact  of  those  predictions;  Dr.  Jerry  Boatz  (AFRL/RZSP)  presented 
work  on  the  design  of  energetic  ionic  liquids;  Dr.  Scott  Fawaz  (USAFA)  presented  work  on 
modeling  fatigue  issues  for  aircraft;  and  Dr.  Andrew  M.  Rappe  (U.  of  Penn.)  discussed  results 
using  computational  techniques  to  explore  the  acoustic  properties  of  complex  ferroelectric 
oxides  for  sonar  applications. 

Attendance  increased  from  400  to  451  attendees  from  last  year's  conference.  There  were 
116  technical  talks,  12  Birds  of  a  Feather  meetings,  7  tutorials,  and  57  posters.  Eighty-four 
presenters  wrote  papers,  an  increase  from  2007.  (Refer  to  the  Appendix  for  a  list  of  papers 
and  posters  presented.) 

The  papers  were  written  to  capture  the  interest  of  scientists  and  engineers  with  general  scientific  backgrounds. 
Papers  covered  unclassified  scientific  research  and  engineering  development  technical  areas  that  address  the  latest 
computational  techniques.  General  topics  included: 


Department  of  Defense 

Proceedings  of  the 

High  Performance  Computing  Modernization  Program 

Users  Group  Conference 
2008 

Solving  the  Hard  Problems 


Seattle,  WA 
July  1 4-July  17,  2008 


User  Advocacy  Group  (UAQ)  g  l  I  -* 

HPCMPO  Outreach  Team  l.Jj'k'tW 


Computational  Fluid  Dynamics  (CFD)  12  papers 

Multi-Physics  (CFD,  CSM,  CCM  Plasma  Physics,  etc.)  12  papers 

Computational  Chemistry  and  Materials  Science  (CCM)  15  papers 

Climate/Weather/Ocean  Modeling  and  Simulation  (CWO)  8  papers 

Signal/lmage  Processing  (SIP)  and  Sensors;  Electronics,  8  papers 

Networking,  and  Systems/C4ISR  (ENS)  and  Testing 

Software  and  Hardware  Infrastructure  9  papers 

Software  Performance  and  Performance  Modeling  9  papers 

Computational  Methods  9  papers 

Education  2  papers 
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The  papers  presented  at  the  UGC  provide  a  good  overview  of  the  unclassified  computational  science  and  engineering 
research  and  development  in  the  DoD  S&T  and  T&E  community.  The  conference  format  and  schedule  provided 
time  for  discussions  among  attendees  during  breaks,  lunch,  and  after  the  day's  presentations. 


Hero  Awards 

Every  year,  the  HPCMP  requests  nominations  from  the  HPCMP  community  for  the  UGC  Hero  Awards.  The  Hero 
Awards  seek  to  recognize  outstanding  individuals  who  support  researchers  and  the  Program  to  make  a  positive 
impact  to  the  DoD  and  the  Warfighter,  contributing  to  the  overall  mission  of  the  HPCMP.  Hero  Awards  recipients 
receive  recognition  at  the  UGC  Hero  Awards  Ceremony. 


Eligibility  extends  to  personnel  within  the  HPCMP  components,  such  as  the  DSRCs,  DREN,  SAS,  and  the  Program 
Office.  The  Hero  Awards  are  given  in  the  following  categories: 

1.  Innovative  Management  -  Demonstrates  creative  business  practices  to  improve  overall  HPCMP  business 
model 

2.  Technical  Excellence  -  Demonstrates  scientific  or  engineering  excellence  using  HPCMP  resources  to  advance 
creative  and  effective  technology 

3.  Long  Term  Sustained  Award  -  Provides  overall  contribution  to  the  science  and  technology  (S&T)  and  test  and 
evaluation  (T&E)  HPCMP  communities  for  the  last  five  years 

4.  Up  &  Coming  within  the  HPCMP  -  Provides  a  contribution  to  the  S&T  and  T&E  HPCMP  communities  and 
recipient  must  have  two  or  less  years  in  the  Program 

In  2008,  the  following  winners  received  HPCMP  Hero  Awards  at  the  UGC  in  Seattle,  WA: 

Innovative  Management  as  chairperson  of  the  NGTS  Source  Selection  Team  -  Mr.  Thomas  Brown,  HPCMPO 
Technical  Excellence  for  management  at  ERDC  MSRC  -  Mr.  Robert  Alter,  ERDC 

Long  Term  Sustained  for  technical  leadership  as  one  of  the  founders  of  HPCMP  -  Dr.  Larry  Davis,  HPCMPO 
Up  &  Coming  as  a  key  member  of  the  AFRL  MSRC  PKINIT  project  -  Mr.  John  Hoffman,  AFRL 

DREN  Networking  and  Security  Conference 


The  2008  DREN  Network  and  Security  Conference  was  held  on  August  11-15,  2008  in  San  Diego,  CA.  This  annual 
conference  was  a  five-day  event  that  showcased  tutorials,  hands-on  technical  sessions,  and  lectures.  A  total  of  262 
people  attended  the  conference.  (Refer  to  the  Appendix  for  a  list  of  presentations.) 


DREN  conference  attendees  included  the  HPCMP  program  managers, 
directors  of  MSRCs,  service  enclaves  on  the  DREN,  and  technical 
and  financial  DREN  users. 

DREN  tutorial  topics  included:  DREN  Network  Visibility  and 
Troubleshooting  with  CNIC  and  Remedy,  Aaron  Gibson  and  Patrick 
Hughes;  Arbor  S-Flow,  Rob  Hartman;  Performance  Tools,  Phil  Dykstra; 

SSG,  Ben  Eater;  Advanced  Linux  Security  (While  Wearing  a  Red  Hat), 

Drew  York;  Introduction  to  Multicast  with  Hands  on  Lab,  Terry  Jones 

and  Stan  Carver;  Comprehensive  Security  Assessment  101,  Drew  York  and  Gerald  Fowler;  Advanced  Multicast  with 
Hands  on  Lab,  Tim  Owen  and  Raul  Fernandez;  CSA  Checklist  Tutorial,  Steve  Simpson;  and  Dr.  Larry  Smarr,  the  Director 
for  the  California  Institute  for  Telecommunications  and  Information  Technology  was  the  Keynote  Speaker. 


DREN  Networking  and  Security  Conference  2008 
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Joint  Educational  Opportunities  for  Minorities  (JEOM) 

The  HPCMP  JEOM  Program  is  an  educational  outreach  effort  that  strives  to  increase  access  to  the  HPC  community 
for  underrepresented  groups  in  the  science,  information  technology,  engineering,  and  mathematics  disciplines.  The 
2008  JEOM  Program  recruited  students  through  HPCMP  program  investigators;  counselors  and  professors  at  colleges 
and  universities  around  the  country;  intern  and  career  Web  sites;  and  MSIs,  such  as  tribal  colleges,  Historically  Black 
Colleges  and  Universities,  and  Hispanic-Serving  Institutions. 

The  JEOM  Program  received  more  than  250  applications  for  36  positions  at  11 
Host  institutions  included: 

Air  Force  Research  Laboratory,  WPAFB,  OH 
Air  Force  Research  Laboratory,  Rome,  NY 
Arctic  Region  Supercomputing  Center,  Fairbanks,  AK 
Army  Research  Laboratory,  Aberdeen,  MD 
Army  Research  Laboratory,  Adelphi,  MD 
Georgia  Institute  of  Technology,  Atlanta,  GA 
HPC  Modernization  Program  Office,  Lorton,  VA 
Iowa  State  University,  Ames,  IA 
Maui  High  Performance  Computing  Center,  Kihei,  HI 
Naval  Air  Warfare  Center  Aircraft  Division,  Patuxent  River,  MD 
Naval  Air  Warfare  Center  Weapons  Division,  China  Lake,  CA 
Space  and  Missile  Defense  Command,  Huntsville,  AL 

The  HPCMP  sponsored  36  interns,  including  8  graduate  students,  3  seniors,  11  juniors,  11  sophomores,  and  3 
freshmen  in  JEOM  internships  for  the  summer.  Over  250  applicants  applied  from  several  universities,  including 
Harvard,  Cornell,  Rutgers,  Grambling,  and  Hampton. 

JEOM  interns  assisted  work  in  many  areas.  The  following  project  descriptions  provide  a  sample  of  their  work: 
characterization  of  DNA-derived  biopolymer  composites  to  optimize  their  performance  in  optoelectronic  devices; 
development  of  a  numerical  model  for  honeycomb  composite  sandwich  panels  capable  of  predicting  the  mechanical 
response  of  beam  components  during  bending  loads;  creation  of  a  simplified  100X  JBI  installation  that  is  convenient 
for  a  user  to  follow  a  basic  set  of  instructions;  creation  of  a  new  benchmark  for  the  Fast  Fourier  Transform  (FFT) 
that  is  quicker  than  the  previous  two  benchmarks;  reduction  of  random  noise  within  images;  optimization  of  a 
missile  defense  scenario;  determination  of  minimum  time-to-target  path  for  a  simple  missile  and  design  of  a  system 
of  lenses;  development  of  baseline  performance  tools  for  various  scenarios,  including  bullet  and  fragment  impact 
against  confined  warheads  and  rocket  motors  using  HPC  assets  to  track  changes  in  performance  (improvements); 
writing  a  parallel  program  in  FORTRAN  using  Message-Passing  Interface  (MPI)  for  the  General  Cartesian  Mesh  Flow 
Solver;  determine  if  the  amount  of  transmission  loss  can  be  estimated  using  reverberation  plots;  and  model  and 
solve  electromagnetism  problems  using  computational  methods. 

JEOM  Summer  Colloquium 

The  JEOM  Summer  Colloquium  for  the  interns  is  designed  as  a  "seminar  for  success"  for  the  interns.  This  year, 
the  three-day  workshop  was  held  in  Seattle,  WA  from  9-11  July  2008.  Interns  received  an  introduction  to  high 
performance  computing,  the  HPCMP,  and  listened  to  technical  talks  from  leaders  in  their  fields,  mentoring  and 


DoD  sites  for  this  1 0-week  program. 
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ethics  in  research.  JEOM  interns  also  attended  the  HPCMP  UGC  from  14-17  July  2008. 
The  UGC  was  the  capstone  to  the  Colloquium. 


The  JEOM  Summer  Colloquium  promotes  interns' 
professional  development  as  scientists,  engineers 
and  researchers  to  build  a  successful  career 
within  the  DoD  RDT&E  community.  The 
following  speakers  discussed  a  variety  of  topics, 
including:  Introduction  to  High  Performance 
Computing  Modernization  Program;  Women 
and  Minorities  in  Information  Technology;  High 
Performance  Computing  in  Test  and  Evaluation;  HPC 
Research;  Aeromechanics  and  Computational  Fluid  Dynamics; 
Science  and  Technology  within  the  DoD  HPC  Community; 
Building  Enzymes  from  Scratch;  Dr.  Rajiv  Berry,  Introduction 
to  Computational  Science;  and  The  Application  of  Computational 
Science  and  High  Performance  Computing  to  the  Design  of  DoD 
Systems. 


University  Workshops 

In  FY  2008,  students  and  faculty  attended  one  of  two  HPCMP-sponsored  workshops: 


Computational  Science  Workshop  for  Underrepresented  Groups 
Computational  Science  and  Engineering  Workshop 

The  Computational  Science  Workshop  for  Underrepresented  Groups  at  the  University 
of  Southern  California  (USC)  in  May  2008  provided  hands-on  learning  experiences 
including  building  systems,  networking  clusters,  programming  languages,  and 
libraries.  The  institute  provided  work  experience  and  an  introduction  to  the  DoD 
supercomputers  and  applications,  to  train  STEM  students  and  faculty  from  HBCUs 
and  MSIs  and  build  an  awareness  about  computational  science  methods  in  STEM 
disciplines.  In  FY  2008,  this  workshop  took  place  at  USC  for  faculty  and  students 
from  North  Carolina  Agricultural  &  Technical  University  (NCA&T),  the  only  historically 
black  university  that  offers  a  doctoral  degree  in  computational  science. 

The  Computational  Science  and  Engineering  Workshop  provided  education  for  faculty 
from  HBCUs  and  MSIs  at  NCA&T  in  June  2008.  The  overall  scope  of  this  workshop 
was  to  provide  faculty  hands-on  training  and  exposure  to  computational  science 
and  engineering  and  high  performance  parallel  scalable  computing.  Approximately 
20  faculty  members  attended  the  workshop  for  professional  development  to  listen 
to  presentations  on  Unix/Linux  operating  systems;  programming  for  HPC;  parallel 
programming  methodologies  and  models;  large-scale  data  analysis;  and  scientific 
visualization. 


63 


HPCMP  Annual  Report  Z008 


National  Defense  Science  and  Engineering  (NDSEG)  Fellowship 
Program 

In  FY  2003,  the  HPCMP  joined  the  NDSEG  Fellowship  Program  and,  in  FY  2004,  began  providing  fellowships  to 
students  interested  in  employing  high  performance  computing  defense-related  science  and  engineering  disciplines. 
The  NDSEG  provides  the  US  with  a  highly  trained  workforce  to  tackle  state-of-the-art  research  projects  in  disciplines 
meeting  the  greatest  payoff  to  national  defense  requirements.  The  HPCMP  focuses  on  helping  students  interested 
in  the  breadth  of  disciplines  related  to  computational  science  and  engineering. 

Between  FY  2003-2008,  the  HPCMP  supported  30  fellowships  through  the  NDSEG  program,  and  continues  to  sponsor 
approximately  10  new  fellows  every  year.  Participating  host  institutions  vary  from  year-to-year.  In  addition,  these 
students  were  invited  to  attend  the  2008  UGC.  Several  fellows  are  highly  involved  in  working  with  DoD  scientists 
and  engineers  and  attended  the  FY  2008  UGC's  plenary  sessions  and  presented  posters  of  their  work  at  the  2008 
UGC  (Refer  to  UGC  section).  In  FY  2008,  the  HPCMP  sponsored  10  new  students  from  the  following  disciplines: 
computer  sciences,  biological  sciences,  mechanical  engineering,  and  electrical  engineering. 

DoD  Institutes  for  Higher  Learning  Program — Cadet  and 
Midshipmen  Summer  Projects 

The  High  Performance  Computing  at  DoD's  Institutions  of  Higher  Fearning  Program  was  established  by  the 
DUSD(S&T)  in  2001.  Its  goal  is  to  provide  HPC  opportunities  to  DoD  Service  academies  and  institutions  of  higher 
learning  in  order  to  prepare  our  future  military  leaders  to  excel  in  tomorrow's  technologically  complex  world. 
Specifically,  the  HPCMP: 

Provides  high  performance  computing  systems  for  DoD  educational  institutions.  (These  systems  are  those 
retired  from  the  DSRCs.) 

Creates  opportunities  for  faculty  and  students  of  these  institutions  to  participate  in  off-site  research,  including 
summer  research  programs  at  laboratories  and  test  centers  where  computational  research  is  performed. 

Enables  faculty  and  student  participation  in  technical  symposia  sponsored  by  the  HPCMP,  including  its  annual 
DoD  Users  Conference. 

Provides  specialized  training  in  high  performance  computing  topics  at  DoD  educational  institutions. 

In  FY  2008,  the  HPCMP  sponsored  18  academy  faculty  members,  cadets,  and  midshipmen  working  13  projects 
submitted  by  the  DoD  laboratories  and  test  centers.  Faculty  and  students  participated  in  the  FY  2008  UGC  and 
several  students  gave  presentations  at  other  scientific  and  technical  forums.  Host  institutions  included: 

Army  Engineering  Research  and  Development  Center,  Information  Technology  Faboratory,  Vicksburg,  MS 
Maui  High  Performance  Computing  Center,  HI 
Naval  Postgraduate  School,  Monterey,  CA 

Opposing  Forces  (OPFOR)  Tech  Unit,  Aviation  &  Foreign  Systems  Division,  Warfighter  Directorate,  Aberdeen 
Test  Center,  Aberdeen  Proving  Ground,  MD 

Cadets  and  midshipmen  assisted  work  in  the  following  areas,  as  a  sample:  Computer  Security;  Computer  Graphics 
and  Visualization;  Playstation  3  Cluster;  Multicore  Programming  Using  the  Playstation  3;  Parallel  Monte  Carlo 
Atmospheric  Radiative  Transfer  Code;  General  Test  Support  with  OPFOR  Tech  Unit;  High  Resolution  Simulations 
of  Aero-optical  Flow  Fields;  Flow  Control  Using  Atmospheric  Plasmas;  High  Resolution  Simulation  of  Full  Aircraft 
Stability  and  Control;  Special  Physics  Projects;  and  Effect  of  Targeting  Pod  on  F-18C  Flowfields. 
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The  goal  of  HPCMP  Oversight  is  to  ensure  the  HPCMP  meets  DoD  guidelines  and  provide  guidance  for  improvement 
to  Program  decision-makers.  The  highest  level  body  responsible  for  oversight  of  HPCMP  is  the  Overarching 
Integrated  Product  Team  Configuration  Steering/Configuration  Steering  Board  (OIPT/CSB)  chaired  jointly  by  Deputy 
Director  of  Defense  (Science  and  Technolgy)  [DUSD(S&T)]  and  Assistant  Secretary  of  Defense,  Networks  and 
Information  Integration  ASD(NII).  HPCMP  Oversight 
gathers  data  and  provides  the  HPCMP 
performance  information  for  the  annual 


metrics  established  by  the  OIPT/CSB. 
This  data,  which  is  obtained  through 
stakeholder  surveys  and  from  HPCMP 
operational  databases,  is  used  not 
only  by  the  OIPT/CSB  for  Program 
oversight,  but  also  by  HPCMP  decision 
makers  for  continuous  improvement  of 
the  Program. 


HPCMP  Metrics 


Oversight 

Mission 

To  ensure  that  the  value  of  HPC  is 
captured  and  successfully 
communicated  to  the  program's 
stakeholders  and  governance 
body  across  the  Department. 


Reporting 


In  Feb  2008,  the  OIPT/CSB/High 
Performance  Computing  Advisory  Panel 
(HPCAP)  discussed  an  overview  of  the 
Program,  status  of  CREATE,  and  established 
metrics,  thresholds,  and  objectives  for  the 
following  year.  During  the  meeting,  the 
OIPT/CSB/HPCAP  reviewed  all  requirements 
and  significant  technical  configuration  changes, 
which  have  the  potential  to  result  in  cost  and 
schedule  impacts  to  the  Program. 


Metrics  include  objective,  thresholds,  and  achievements  from  all 

facets  of  the  HPCMP:  DREN,  Program  Security,  HPC  Centers,  Contracts/Procurement,  HPC  Software  Applications 
Support,  and  Resource  Management/System  Utilization  Tracking.  The  FY  2008  HPCMP  metrics,  with  results  that 
show  the  high  achievement  record  of  the  HPCMP,  are  shown  on  the  next  page. 
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Output/Outcome  Measure 

Objective 

Threshold 

Achieved 

Defense  Research  and  Engineering  Network  (DREN) 

Total  aggregate  bandwidth  delivered  to  the  user  community 

29  Gbps 

25.9  Gbps 

30.6  Gbps 

Customer  Satisfaction:  Users/Customers  rating  DREN  (on  1-5 
scale) 

5 

3 

4 

Program  Security 

Number  of  annual  Comprehensive  Security  Assessments 
completed 

MSRCs  &  ADCs 
yearly;  new 
SDREN  100% 

DoD  guidelines 
(once  every  three 
years  or  major 
configuration 
change) 

100% 

Cat  1  &  Cat  2  security  incidents  on  PIPC  resources  as  percent  of 
total  number  of  events 

0 

30  incidents  or 
0.0002% 

0  Cat  1  and 

2  Cat  2 

High  Performance  Computing  (HPC)  Centers 

Value  Metric  (Sum  of  work  accomplished  weighted  by  importance 
of  work)  -  Operational  utility  metric  (Flabu  Equivalent-years) 

75%  increase 

25%  increase 

107% 

PIPCMP  system  availability 

98% 

96% 

97% 

Customer  Satisfaction:  Users  rating  PIPC  Centers  (on  1-5  scale) 

5 

3 

3.8 

Contracts/Procurement 

Acquire  the  best  commercially  available  high-end  HPC  capability  — 
Procurement  effectiveness  metric  (Habu  Equivalent-years) 

75%  increase  over 
previous  year 

25%  increase  over 
previous  year 

49% 

Acquire  (including  Congressional  Adds)  the  best  commercially 
available  high-end  HPC  capability  (Habu  Equivalent-years) 

75%  increase  over 
previous  year 

25%  increase  over 
previous  year 

64% 

Provide  (including  Congressional  Adds)  the  best  commercially 
available  high-end  HPC  capability  (Habu  Equivalent-years) 

75%  increase  over 
previous  year 

25%  increase  over 
previous  year 

69.50% 

Customer  Satisfaction:  Users  rating  HPC  systems  (on  1-5  scale) 

5 

3 

3.8 

HPC  Software  Applications  Support 

Percent  of  successful  annual  Institute  evaluations  accomplished 

100% 

80% 

100% 

Percent  of  successful  Software  Portfolio  Alpha/Beta  tests 
accomplished 

100% 

80% 

100% 

^Number  of  acquisition  programs  introduced  to  CREATE  tools 

— 

— 

5 

^Number  of  acquisition  programs  continuing  to  use  CREATE  tools 

— 

— 

— 

Customer  Satisfaction:  Impact  of  HPC  Software  to  RDT&E  (on  1-5 
scale) 

5 

3 

4.4 

Resource  Management/System  Utilization  Tracking 

Percent  of  non-real-time  system  requirements  satisfied 

75% 

25% 

50% 

Percent  of  real-time  system  requirements  satisfied 

50% 

15% 

17% 

Number  of  Challenge  Project  reallocations 

3 

1 

3 

HPCMP  Impact  Examples 

Customer  Satisfaction:  Users/RDT&E  activities  rating  HPCMP  (on 

1-5  scale) 

5 

3 

4.1 

*0n  12  February  2009,  the  CSB/OIPT/FIPCAP  proposed  these  two  output/outcome  measures.  The  PIPCAP  will  report  this  data  in  FY  2009. 
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Satisfaction  Surveys 

Customer  User  Satisfaction  Survey  (C-USS) 

The  Customer  User  Satisfaction  Survey  (C-USS)  provides  feedback  from  the  user  community  to  support  the  ongoing 
operational  assessment  of  the  HPCMP.  The  HPCMPO  conducts  user  surveys  as  an  aid  in  major  decision-making 
to  distribute  resources  and  continually  improve  the  HPC  investments.  Additionally,  the  HPCMPO  reports  survey 
results  to  DUSD(S&T)  and  OASD(NII). 

Each  year,  one  third  of  the  total  users  of  the  HPCMP  are  surveyed.  In  2008,  there  were  over  1,434  users  and  3,963 
active  accounts.  The  Oversight  component  collected  information  for  the  C-USS  between  25  March  and  25  April 
2008.  The  survey  was  distributed  to  579  users  and  the  HPCMP  received  269  responses,  43  percent  participation, 
giving  the  Program  a  97  percent  confidence  level  with  a  +/-  5%  margin  of  error. 

C-USS  Highlights 

Survey  results  provided  insight  and  suggestions  to  meet  HPC  customer  needs.  Questions  in  the  following  areas,  such 
as  DREN  and  HPC  Centers,  provide  the  Program  with  an  assessment  of  current  performance  and  target  opportunities 
for  improving  customer  product  delivery.  In  the  following  graph,  the  threshold  for  all  areas  is  3.0. 


HPC  Centers  DREN  Contracts/Procurements  PET 

2008  C-USS  Results 


S/AAA  Survey 

The  HPCMP  community  includes  approximately  100  Service/ Agency  approval  Authority  (S/AAA)  to  represent 
every  organization.  The  role  of  the  S/AAA  is  essential  to  providing  the  computational  resources  to  DoD  scientists 
and  engineers.  The  S/AAA  serves  as  the  liaison  between  the  Program  and  the  user.  They  allocate  Service/ Agency 
controlled  CPU  hours  on  HPCMP  systems;  provide  day-to-day  linkage  with  customer  organizations;  the  HPCMP 
and  other  S/AAAs;  ensure  Project  Leaders  and  authorized  users  are  performing  work  in  support  of  DoD;  implement 
requirements  surveys;  resource  allocation;  resource  monitoring  and  resource  reallocation;  and  provide  guidance  to 
users  on  which  HPC  assets  are  appropriate  for  their  projects. 
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The  S/AAA  survey  is  administered  every  other  year,  including  FY  2008.  In  FY  2008,  56  of  100  S/AAAs  participated  in 
this  survey.  S/AAA  respondents  with  two  or  more  years  S/AAA  experience  made  up  76  percent.  S/AAA  respondents 
with  one  to  two  years  S/AAA  experience  made  up  16  percent,  while  S/AAA  respondents  with  less  than  one  year 
S/AAA  experience  made  up  8  percent.  Overall,  the  S/ AAA's  were  very  satisfied  with  the  workshops,  their  role,  and 
all  aspects  of  the  duties  as  an  S/AAA. 


Resource  Allocation  User  Liaison  User  Organization  Requirements  Account  Annual 

Manager  Administration  Consultant  Advisor  Coordinator  Management  Reports 

S/AAA  Survey  Results 
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Continuing  FY  2008  Challenge  Projects 

Computational  Chemistry  Modeling  of  the  Atmospheric  Fate  of  Toxic  Industrial  Compounds 

Project  Leaders:  Steven  Bunte  and  Margaret  Hurley  (ARL)  and  David  Curran  (AFTAC) 

Sponsor:  Army 

This  project  used  state-of-the-art  computational  quantum  chemistry/quantum  dynamics  models  to  compute  rate  constants  of 
compounds  in  the  atmosphere.  The  addition  of  finite  chemistry  will  result  in  the  improved  models,  resulting  in  a  more  realistic 
representation  of  the  contamination  zone. 

Computational  Simulations  of  Combustion  Chamber  Dynamics  and  Hypergolic  Propellant  Chemistry  for  Selectable 
Thrust  Liquid/Gel  Rocket  Engines 

Project  Leader:  Michael  Nusca  (ARL) 

Sponsor:  Army 

The  project  applied  high  performance  computing  to  accelerate  the  development  of  hypergolic  propulsion  systems  (one 
that  employs  a  pumpable  liquid  or  gel  fuel  and  a  pumpable  oxidizer  that  ignites  spontaneously  when  mixed  at  ambient 
temperatures  and  pressures  for  tactical  missiles).  Computational  fluid  dynamics  was  employed  to  model  the  chemically 
reacting  flow  within  a  system’s  combustion  chamber,  and  computational  chemistry  is  employed  to  characterize  propellant 
physical  and  reactive  properties.  The  US  Army  will  use  the  project’s  results  to  develop  the  Impinging  Stream  Vortex  Engine 
(ISVE)  for  tactical  missile  applications. 

DARPA  Underwater  Express 

Project  Leader:  Robert  Kunz,  Pennsylvania  State  University 
Sponsor:  Office  of  Naval  Research 

The  project  presented  several  large  scale  multiphase  Navier-Stokes  simulations  of  flows  of  relevance  to  the  Defense 
Advanced  Research  Projects  Agency  (DARPA)  Underwater  Express  Program.  The  project  explored  the  role  of  free-surface 
proximity  to  the  physics  and  performance  of  high  speed  supercavitating  vehicle  components  under  the  DARPA  Underwater 
Express  Program. 

Decision  Support  for  Seismic  and  Acoustic  Sensors  in  Urban  Terrain 

Project  Leader:  Stephen  Ketcham,  (ERDC) 

Sponsor:  Army 

This  project  highlighted  statistical  scattering  measures  from  high-performance  computations  of  urban  sound  propagation  and 
concluded  that  these  wave-field  measures,  which  are  impractical  to  obtain  experimentally,  provide  a  unique  understanding  of 
urban  sound  scattering  and  produce  information  that  can  be  readily  integrated  into  sensor-placement  decision-support  tools. 

Direct  Numerical  Simulations  of  Active  Control  for  Low-Pressure  Turbine  Blades 

Project  Leader:  Hermann  Fasel,  University  of  Arizona 
Sponsor:  Air  Force  Office  of  Scientific  Research 

One  of  our  objectives  for  FY  2008  was  to  continue  controlled  and  uncontrolled  flow  simulations  to  obtain  better  averaged  data 
and  converged  turbulent  statistics.  The  turbulent  statistics  allow  a  comparison  with  experimental  data  assessment  of  the 
physical  grid  resolution.  The  DoD  can  drastically  improve  efficiency  of  jet  engines,  increase  reduction  in  engine  component 
weight,  and  increase  range  of  flight  vehicles. 

Direct  Quantum  Mechanical  Simulations  of  Shocked  Energetic  Materials  Supporting  Future  Force  Insensitive 
Munitions  (IM)  Requirements 

Project  Leader:  William  Mattson  (ARL) 

Sponsor:  Army 

This  project  provides  a  quantum  mechanical  characterization  of  the  properties  and  behavior  of  shocked  energetic  materials 
under  a  variety  of  initiating  conditions.  This  work  will  provide  critical  information  that  supports  DoD  Insensitive  Munitions 
initiatives  and  the  need  for  multi-purpose  energetics  for  Urban  Operations. 

Integrated  Analysis  of  Scramjet  Flowpath  with  Innovative  Inlets 

Project  Leader:  Datta  Gaitonde  (AFRL) 

Sponsor:  Air  Force 

During  the  second  year  of  this  effort,  the  project  employed  combustion  simulations  to  provide  significant  insight  into  the 
relative  impact  of  flow  distortion  arising  in  different  inlet  designs.  The  results  represent  a  high-fidelity  effort  to  employ 
simulations  to  guide  the  development  of  high-speed  vehicles  capable  of  survivable  atmospheric  (air-breathing)  flight, 
facilitating  conventional  munitions  strike  with  adaptive  target  selection,  including  ability  to  plan  en  route  to  the  mission. 
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Continuing  FY  2008  Challenge  Projects 

Integration  of  Simulation  and  Test  for  Strike  Aircraft  Design  and  Development 

Project  Leader:  Bradford  Green,  NAVAIR 
Sponsor:  Navy 

This  project  intends  to  improve  the  performance  of  the  aircraft  by  focusing  on  flow  control  to  attach  the  boundary  layer  on 
the  wing  and  increase  lift  on  the  aircraft  at  low  speeds.  Two  wind-tunnel  tests  provide  wind-tunnel  data  for  comparison  of  the 
CFD  results.  CFD  can  assist  the  DoD  in  providing  estimates  of  the  impact  of  changes  on  aircraft  forces  and  moments. 

Large-Scale  Deterministic  Predictions  of  Nonlinear  Ocean  Wave-Fields 

Project  Leader:  Dick  K.P  Yue,  Massachusetts  Institute  of  Technology 
Sponsor:  Office  of  Naval  Research 

Using  the  direct  simulation  capability,  the  project  investigated  the  statistics  of  the  nonlinear  wavefield.  By  comparing 
the  results  with  the  field  measurements,  our  model  showed  to  be  a  reliable  and  useful  tool  capable  of  predicting  the 
key  characteristics  of  nonlinear  ocean  surface  wavefields.  An  accurate  spatial-temporal  prediction  of  large-scale  ocean 
wavefields  is  essential  for  environmental  input  to  the  design  and  operation  of  Navy  ships. 

Modeling  of  Mine  Countermeasure  Dart  Dispense:  Multiple-Body  Six  Degree-of-Freedom  (6-DOF)  Computational  Fluid 
Dynamics  (CFD)  with  Collisions 

Project  Leader:  Gary  Prybyla  (NSWC) 

Sponsor:  Navy 

To  support  the  development  and  evaluation  of  dart  dispense  concepts,  the  project  developed  a  set  of  computational  tools 
to  accurately  predict  the  dispersion  pattern,  velocities,  and  orientations  of  darts.  The  Navy  and  Marine  Corps  will  use  the 
capability  to  rapidly  clear  mines  in  the  surf-zone  and  beach-zone  regions  to  permit  rapid,  effective,  transition  from  the  sea  to 
land. 

Polynitrogen/Nanoaluminum  Surface  Interactions 

Project  Leader:  Jerry  Boatz  (AFRL) 

Sponsor:  Air  Force 

Investigation  of  the  interactions  of  energetic  high-nitrogen  compounds  with  ultrafine  or  nanophase  aluminum  particles  is  a 
topic  of  current  DoD  interest  with  potential  applications  in  rocket  and  missile  propulsion.  The  project  examined  first-principles 
density  functional  theory  (DFT)  calculations  using  the  generalized  gradient  approximation  (GGA)  to  study  the  adsorption  of  a 
series  of  all-nitrogen  and  high-nitrogen  compounds  of  increasing  sizes  and  complexity.  The  overall  impact  of  the  proposed 
work  is  cost  effective  and  gains  more  reliable  access  to  space  via  development  of  advanced  energetic  materials  for  rocket 
propulsion  applications. 

Signature  Evaluation  for  Thermal  Infrared  Countermine  and  IED  Detection  Systems 

Project  Leader:  John  Peters  (ERDC) 

Sponsor:  Army 

The  results  of  the  simulations  were  used  to  determine  the  optimal  times  of  day  for  thermal  imagery  based  on  using  relative 
automated  target  recognition  (ATR)  performance  as  an  objective  measure  and  later  confirmed  in  field  tests.  The  DoD  can  use 
sensor  simulations  to  optimize  mission  parameters  and  reduce  false  alarms  from  ATR  algorithms. 

Unsteady,  Multidisciplinary  Rotorcraft  Simulations  for  Interactional  Aerodynamics 

Project  Leader:  Mark  Potsdam  (ARDEC) 

Sponsor:  Army 

The  project  used  state-of-the-art,  large-scale,  parallel,  moving  body  computations  to  view  flow  field  interactions  on  complete 
aircraft  geometries,  leading  to  possible  investigation  of  adverse  phenomena  and  aircraft  performance.  Current  DoD  weapons 
systems  will  benefit  directly  from  this  work,  including  the  MH-47  Chinook  and  UFI-60  Blackhawk.  Both  exhibit  complex,  3-D, 
interactional  aerodynamics. 
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Continuing  FY  2008  DHPIs 

Advanced  LINUX  Cluster  Optimized  to  Support  the  Interactive  Joint  Futures  Laboratory 

Project  Leader:  Jack  Wagner,  JFCOM/J7  &  J9,  Suffolk,  VA 
Sponsor:  JFCOM 

This  uniquely  configured  256-node  Linux  cluster,  with  a  General  Purpose  GPU  on  each  4-core  node,  enables  JFCOM  to  model 
urban  battlespaces  with  10M  independent  agents,  providing  a  cost-effective  and  socially-acceptable  way  to  look  at  the  use  of 
new  tactics  and  sensors  for  future  conflicts  in  urban  areas. 

Automated  Generation  of  Nomografs  for  Urban  Chemical  Dispersion  Scenarios 

Project  Leader:  Jay  Boris,  NRL,  Washington,  DC 
Sponsor:  Navy 

This  project  allows  real-time  analysis  of  urban  chemical  dispersion  through  the  prior  calculation  of  events  under  many  different 
conditions  in  order  provide  a  database  of  possibilities  providing  live,  near  CFD-quality  results  instantly  in  a  crisis  for  any 
complex  geometry  region. 

Joint  Ensemble  Forecast  System,  AFWA,  Offutt  AFB,  NE  and  FNMOC,  Monterey,  CA 

Project  Leader:  Capt.  Tim  Nobis,  AFWA,  Offutt  AFB,  NE  and  Michael  Sestak,  FNMOC,  Monterey,  CA 
Sponsor:  Air  Force  and  Navy 

This  project  provides  accurate  meteorological  and  oceanographic  intelligence  to  air,  ground  and  fleet  decision-makers  to 
facilitate  a  more  realistic  evaluation  of  the  potential  environmental  impact  of  weather  on  combat  operations. 

HPC  Upgrade  of  the  Joint  Warfare  Systems  Server 

Project  Leader:  Al  Sweetser,  OSD(PA&E),  Washington,  DC 
Sponsor:  OSD 

This  project  supports  campaign-level  analysis,  combining  asset  mobility,  theater  logistics,  and  joint  service  warfighting,  to 
provide  a  robust  simulation  of  complex  military  scenarios. 

PS3  Cluster  for  Neuromorphic  Computing 

Project  Leader:  Richard  Linderman,  AFRL/RI,  Rome,  NY 
Sponsor:  Air  Force 

This  project  provides  cell-based  computing  via  a  cluster  of  336  PS3  gaming  consoles.  Applications  benefiting  from  the  >10X 
S/performance  advantage  of  this  system  include:  1)  large  scale  simulations  of  neuromorphic  models,  2)  GOTCHA  radar  video 
SAR  for  wide-area  persistent  surveillance,  and  3)  real-time  PCID  image  enhancement  for  space  situational  awareness. 
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Publications 

UGC  2008  Papers 

Computational  Fluid  Dynamics 
(CFD) 

A  New  Approach  to  Streambed 
Modeling  and  Simulation  Using 
Computational  Fluid  Dynamics,  J.  Allen, 
D.  Smith,  O.  Eslinger,  and  M.  Valenciano 

Active  Separation  Control  for  Lifting 
Surfaces  at  Low-Reynolds  Number 
Operating  Conditions,  A.  Gross,  W. 
Balzer,  and  H.F.  Fasel 

An  Undergraduate  Computational 
Aerodynamics  Curriculum,  K.  Bergeron, 
R.  Cummings,  D.  McDaniel,  Capt  R. 
Decker,  Maj  J.  Freeman,  Capt  C.  Hoke,  J. 
Seidel,  and  S.  Morton 

Free-Surface  Proximity  Effects  in 
Developed  and  Super-Cavitation,  M. 
Kinzel,  J.  Lindau,  and  R.  Kunz 

High-Fidelity  Computations  of  Moving 
and  Flexible  Wing  Sections  with 
Application  to  Micro  Air  Vehicles,  R. 
Gordnier,  M.  Visbal,  and  M.  Galbraith 

High-Resolution  Simulations  of 
Nonlinear  Internal  Gravity  Waves  in 
the  South  China  Sea,  O.  Fringer  and  Z. 
Zhang 

Modeling  of  Mine  Countermeasure  Dart 
Dispense,  G.  Prybyla,  M.  Neaves,  and 
W.  Dietz 

Numerical  Investigation  of  Internal 
and  External  Three-Dimensional  Flow 
Separation,  A.  Gross,  R.  Jacobi,  S. 

Wernz,  and  H.F.  Fasel 

Numerical  Simulation  of  Non-Resonant 
Cavity  Flow,  C.  Wagner  and  S.  Slimon 

Ocean  Wave  Prediction  Using  Large- 
Scale  Phase-Resolved  Computations,  W. 
Xiao,  L.  Henry,  Y.  Liu,  K.  Hendrickson, 
and  D.K.P.  Yue 

Prediction  of  High-Amplitude  Forces 
During  Propeller  Crashback,  P.  Chang, 

III,  M.  Ebert,  J.  Shipman,  and  K.  Mahesh 

The  High  Speed  Sea  Lift  (HSSL)  Ships 
Challenge  Effort,  J.  Gorski,  R.  Miller,  P. 
Carrica,  M.  Kandasamy,  and  F.  Stern 


Multi-Physics  (CFD,  CSM,  CCM, 
Plasma  Physics) 

Application  of  3-D,  Unsteady  Navier- 
Stokes  Simulation  to  Chemical  Oxygen- 
Iodine  Laser  Technology  Development, 

T.  Madden 

Characterization  of  Hot-Wire  Detonators 
Using  Analytical  Modeling  and 
Computational  Tools,  M.  Lambrecht,  C. 
Baum,  E.  Schamiloglu,  and  K.  Cartwright 

Combustion  Chamber  Fluid  Dynamics 
and  Hypergolic  Gel  Propellant 
Chemistry  Simulations  for  Selectable 
Thrust  Rocket  Engines,  M.  Nusca,  C-C. 
Chen,  and  M.  McQuaid 

Comparisons  of  Two-Fluid  Plasma 
Models,  B.  Srinivasan  and  U.  Shumlak 

Fast  and  Reliable  Solution  of  GDoF- 
Problems  on  NAVO/BABBAGE  and 
AFRL/HAWK  Systems,  S.  Fawaz  and  B. 
Andersson 

Generation  of  Aerodynamic  Coefficients 
using  Time-Accurate  CFD  and  Virtual 
Fly-Out  Simulations,  J.  Sahu,  S.  Silton,  J. 
DeSpirito,  K.  Heavey,  and  M.  Costello 

Integrated  Analysis  of  Scramjet  Flowpath 
with  Innovative  Inlets,  D.  Gaitonde,  F.J. 
Malo-Molina,  H.  Ebrahimi,  and  D.  Risha 

MHD  Turbulence  Studies  Using 
Lattice  Boltzmann  Algorithms-Physical 
Simulations  Using  9,000  Cores  on  the 
Air  Force  Research  Laboratory  HAWK 
Supercomputer,  G.  Vahala,  J.  Yepez,  M. 
Soe,  L.  Vahala,  J.  Carter,  and  S.  Ziegler 

Multidisciplinary  Modeling  of  the  CH-47 
Helicopter  with  CFD/CSD  Coupling 
and  Trim,  A.  Dimanlig,  H-A.  Saberi, 

E.  Meadowcroft,  R.  Strawn,  and  M. 
Bagwhat 

Regions  of  Validity  for  the  1 0-Moment, 
Two  Fluid  Plasma  Model,  R.  Lilly  and  U. 
Shumlak 

Virtual  Prototyping  of  Directed  Energy 
Weapons,  M.T.  Bettencourt,  K.L. 
Cartwright,  A.D.  Greenwood,  T.P. 
Fleming,  M.D.  Haworth,  N.P.  Lockwood, 
and  P.J.  Mardahl 

Vulnerability  of  Structures  to  Weapons 
Effects,  J.  Baylot,  S.  Akers,  J.  O'Daniel,  B. 
Armstrong,  and  R.  Weed 


Computational  Chemistry  and 
Materials  Science  (CCM) 

Ab-Initio  Molecular  Dynamics 
Simulations  of  Molten  Ni-Based 
Superalloys,  C.  Woodward,  M.  Asta,  D. 
Trinkle,  J.  Lill,  and  S.  Angioletti-Uberti 

Calculations  of  Lithium+  Carborane 
Complexes,  J.  Vacek,  J.  Chocholousova, 
and  J.  Michl 

Critical  Carbon  Nanotube  Length  in 
Fibers,  C.F.  Cornwell,  D.  Majure,  R. 
Haskins,  N.J.  Lee,  R.  Ebeling,  R.  Maier, 

C.  Marsh,  A.  Bednar,  R.  Kirgan,  and  C.R. 
Welch 

Crystal  Structures  from  Nonempirical 
Force  Fields,  R.  Podeszwa,  B.  Rice,  F. 

Rob,  and  K.  Szalewicz 

Direct  Quantum  Mechanical  Simulations 
of  Shocked  Energetic  Materials 
Supporting  Future  Force  Insensitive 
Munitions  (IM)  Requirements,  W. 
Mattson,  R.  Balu,  and  B.  Rice 

Design  of  Energetic  Ionic  Liquids,  J. 
Boatz,  M.  Gordon,  G.  Voth,  and  S 
Hammes-Schiffer 

Large-Scale  Atomic/Molecular 
Massively  Parallel  Simulator  (LAMMPS) 
Simulations  of  the  Effects  of  Chirality 
and  Diameter  on  the  Pullout  Force  in  a 
Carbon  Nanotube  Bundle,  D.L.  Majure, 
R.W.  Haskins,  N.J.  Lee,  R.M.  Ebeling, 

R.S.  Maier,  C.P.  Marsh,  A.J.  Bednar,  R.A. 
Kirgan,  C.R.  Welch,  and  C.F.  Cornwell 

Membrane  Insertion  Profiles  of  Peptides 
Probed  by  Molecular  Dynamics 
Simulations,  l-C.  Yeh,  M.  Olson,  M.  Lee, 
and  A.  Wallqvist 

Modeling  of  Materials  for  Naval  SONAR, 
Pollution  Control  and  Nonvolatile 
Memory  Application,  J.  Bennett,  I. 
Grinberg,  Y-H.  Shin,  and  A.M.  Rappe 

Multiscale  Simulations  of  High 
Performance  Capacitors  and 
Nanoelectronic  Devices,  J.  Bernholc,  J. 
Jiang,  V.  Ranjan,  L.  Yu,  M.  Buongiorno 
Nardelli,  and  W.  Lu 

Novel  Mechanism  for  the  Dissociation 
of  H20  and  the  Diffusion  of  O  and 
H  Along  the  0AI2O3  (0001 )  Surface, 

J.  Synowczynski,  J.  Andzelm,  and  D. 
Vlachos 

Performance  of  DFT  Methods  in  the 
Calculation  of  Optical  Spectra  of 
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Chromophores,  J.  Andzelm,  A.  Rawlett, 

J.  Dougherty,  N.  Govind,  and  R.  Baer 

PIPA:  A  High-Throughput  Pipeline  for 
Protein  Function  Annotation,  C.  Yu,  V. 
Desai,  N.  Zavaljevski,  and  J.  Reifman 

Polynitrogen/Nanoaluminum  Surface 
Interactions,  J.  Boatz  and  D.  Sorescu 

Structure  and  Dynamics  of  Squalane 
Films  on  Solid  Surfaces,  M.  Tsige  and  S. 
Patnaik 

Climate/Weather/Ocean 
Modeling  and  Simulation  (CWO) 

Dedicated  High  Performance  Computer 
Project  Investment  (DHPI)  for  the  Fleet 
Numerical/Air  Force  Weather  Agency  - 
The  Navy  Side,  M.  Sestak,  C.  Bishop,  T. 
Holt,  J.  Nachamkin,  S.  Chen,  J.  McLay, 
and  J.  Doyle 

High-Resolution  Simulations  and 
Atmospheric  Turbulence  Forecasting,  J. 
Werne,  D.  Fritts,  L.  Wang,  T.  Lund,  and 

K.  Wan 

Multi-Scale  Predictability  of  High-Impact 
Upper  Tropospheric  Ice  Clouds  for  Air 
Force  Platforms,  A.  Mahalov 

Multi-scale  Predictability  of  High-Impact 
Weather  in  the  Battlespace  Environment, 
J.  Doyle,  C.  Reynolds,  C.  Bishop,  J. 
Goerss,  T.  Holt,  and  J.  McLay 

Numerical  Exploration  of  the  Stable 
Atmospheric  Boundary  Layer,  B. 

MacCall,  P.  Haines,  E.  Measure,  D. 
Marlin,  W-Y.  Sun,  W-R.  Hsu,  and  D. 
Grove 

Towards  the  Development  of  an 
Operational  Mesoscale  Ensemble 
System  for  the  DoD  Using  the  WRF- 
ARW  Model,  T.  Nobis,  E.  Kuchera,  S. 
Rentschler,  S.  Rugg,  J.  Cunningham,  C. 
Synder,  and  J.  Hacker 

Use  of  HPC  to  Provide  Operational 
Mesoscale  Meteorological  Support  for 
ATEC  Test  Ranges,  J.  Pace,  E.  Astling, 

S.  Halvorson,  Y.  Liu,  T.  Betancourt,  J. 
Hacker,  J.  Knievel,  S.  Swerdlin,  and  T. 
Warner 

Wave  Information  Studies  (WIS)  Pacific 
Regional  Hindcast,  B.  Tracy  and  D. 
Spindler 


Signal/Image  Processing  (SIP)  and 
Sensors;  Electronics,  Networking, 
and  Systems/C4ISR  (ENS)  and 
Testing 

Comparison  of  Turbo  Decoder  and 
Packet  Acquisition  Error  Rates  in 
Frequency-Hop  Spread-Spectrum- 
Systems  in  Partial-Band  Interference,  E. 
Huang  and  F.  Block 

Development  of  Biological  Warfare 
Sensors  Using  High  Performance 
Computer  Systems,  M.  Lanzagorta,  J. 
Eversole,  and  W.  Anderson 

Improved  Parallel  3D  FDTD  Simulator 
for  Photonic  Crystal,  J.  Ayubi-Moak,  S. 
Goodnick,  D.  Stanzione,  G.  Speyer,  and 
P.  Sotirelis 

PVTOL:  Providing  Productivity, 
Performance,  and  Portability  to  DoD 
Signal  Processing  Applications  on 
Multicore  Processors,  H.  Kim,  E. 
Rutledge,  S.  Sacco,  S.  Mohindra,  M. 
Marzilli,  J.  Kepner,  R.  Haney,  J.  Daly,  and 
N.  Bliss 

Scattering  of  Seismic  Waves  by  Shallow 
Building  Foundations  Using  High-Order 
FEM,  M.  Parker,  S.  Ketcham,  and  S.  Dey 

Scattering  of  Urban  Sound  Energy  from 
High-Performance  Computations,  S. 
Ketcham,  M.  Parker,  H.  Cudney,  and 
D.K.  Wilson 

Signature  Evaluation  for  Thermal 
Infrared  Countermine  and  IED  Detection 
Systems,  J.  Peters,  S.  Howington,  O. 
Eslinger,  J.  Fairley,  J.  Ballard,  R.  Goodson, 
and  V.  Carpenter 

Validating  Simulations  of  Acoustic 
Propagation  in  Complex  Terrain,  H. 
Cudney,  S.  Ketcham,  D.  Albert,  and  M. 
Parker 

Software  and  Hardware 
Infrastructure 

Asymmetric  Core  Computing,  J.  Clarke, 
D.  Shires,  J.  Vines,  and  E.  Mark 

Coprocessor  Computing  with  FPGA  and 
GPU,  S.J.  Park,  D.  Shires,  and  B.  Henz 

HPCC  Support  to  Campaign  Level 
Analysis  "HPCC  Solving  the  Problem",  S. 
Barnes,  LTC  J.  Crino,  and  LtCol  T.  Smetek 

Improvements  to  Multiple  Path  Secure 
Copy,  B.  Guilfoos,  L.  Humphrey,  and  J. 
Unpingco 


Real-Time  Hardware-in-the-Loop  Testing 
with  Common  Simulation  Framework,  J. 
Gardiner 

The  Computational  Science  Environment 
(CSE),  J.  Renteria  and  E.  Mark 

User  Friendly  High  Productivity 
Computational  Workflows  Using  the 
VISION/HPC  Prototype,  J.  Unpingco 

Using  Mitrion-C  to  Implement  Floating- 
Point  Arithmetic  on  a  Cray  XD1 
Supercomputer,  K.  Liu,  C.  Cameron,  and 
A.  Sarkady 

Visualization  of  Time-Varying  Features, 
M.J.  Mohammadi-Aragh,  S.  Ziegeler,  J. 
Martin,  and  R.  Moorhead 

Software  Performance  and 
Performance  Modeling 

Enabling  High  Productivity  Computing 
Through  Virtualization,  J.C.  Chaves 

High  Performance  Information 
Management  for  HPC  Parallel 
Computing,  G.  Ramseyer,  S.  Emeny,  D. 
Fitzgerald,  R.  Linderman,  S.  Tucker,  and 
S.  Spetka 

High  Productivity  Languages  for  Parallel 
Programming  Compared  to  MPI,  S. 
Spetka,  H.  Hadzimujic,  S.  Peek,  and  C. 
Flynn 

Improving  Parallel  Code  Performance 
for  Systems  with  Dual-Core  Processors, 

S.  Emeny,  S.  Spetka,  G.  Ramseyer,  and  R. 
Linderman 

Modeling  Mixtures  of  Different  Mass 
Ultracold  Atoms  in  Optical  Lattices:  An 
Illustration  of  High  Efficiency  and  Linear 
Scaling  on  the  Cray  XT4  via  a  Capability 
Applications  Project  at  ERDC,  J.K. 
Freericks 

Observing  Parallel  Phase  and  I/O 
Performance  Using  TAU,  S.  Shende,  A. 
Malony,  A.  Morris,  and  D.  Cronk 

Optimization  and  Parallelization  of  DFT 
and  TDD  FT  in  GAMESS  on  DoD  HPC 
Machines,  M.  Lasinski,  N.  Romero,  A. 
Yau,  G  Kedziora,  J-P.  Blaudeau,  and  S. 
Brown 

Performance  Evaluation  of  the  Multi- 
Language  Helios  Rotorcraft  Simulation 
Software,  A.  Wissink  and  S.  Shende 

Performance  Modeling  and  Mapping 
of  Sparse  Computations,  N.  Bliss,  S. 
Mohindra,  and  U-M.  O'Reilly 
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Computational  Methods 

A  Scalability  Study  on  Multicore  Cluster 
Systems  of  an  AFRL  Radar  Frequency 
Tomographiy  Imaging  Code  Written  in 
MATLAB®  for  Parallel  Execution  using 
Star-P®,  B.  Elton  and  K.  Magde 

Early  Experiences  withi  Algorithms 
Optimizations  on  Clusters  of  Playstation 
3%  R.  Linderman 

Exploring  New  Architectures  in 
Accelerating  CFD  for  Air  Force 
Applications,  J.  Dongarra,  G.  Peterson,  S. 
Tomov,  J.  Allred,  V.  Natoli,  and  D.  Richie 

Multicloud  Convergence  Acceleration 
for  Complex  Applications  on  Arbitrary 
Grids,  A.  Katz  and  A.  Jameson 

Paradigms  for  Parallel  Computation,  Gil 
Speyer,  Natalie  Freed,  Richard  Akis,  and 
Dan  Stanzione 

Parallel  Implementation  of  Certain 
Robust  Regression  Methods  Using  Lazy 
Evaluation  in  Python,  J.  Unpingco 

Progress  in  Applying  HPC  to  Support 
Operational  Use  of  CT-Analyst,  G. 
Patnaik,  J.  Boris,  K.  Obenschain,  R. 
Rosenberg  W.  Anderson,  and  M.  Xu 

Some  Comparative  Benchmarks 
for  Linear  Algebra  Computations  in 
MATLAB  and  Scientific  Python,  J. 
Unpingco 

Task  and  Conduit  Framework  for  Multi- 
Core  Systems,  S.  Mohindra,  J.  Daly,  R. 
Haney,  and  G.  Schrader 

Education 

Educating  the  Educator:  High 
Performance  Computing  Training 
Workshop  for  Faculty  from 
Under-Represented  and  Minority 
Serving  Institutions,  R.  Mohan,  N. 
Radhakrishnan,  A.  Kelkar,  and  V.  Thomas 

Women  and  Minorities  in  Information 
Technology,  V.  Ross  and  V.  Thomas 


UGC  2008  Poster 
Presentations 

CREATE/HSAI/Portfolio 

Computational  Research  and 
Engineering  Acquisition  Tools 
and  Environment,  Doug  Post  and 
the  CREATE  Team  (K.  Hill,  D.  van 
Veldhuizen,  G.  Zelinski,  AFRL;  S. 

Arevalo,  T.  Blacker,  D.  Fisher,  P.  Genalis, 
A.  Harris,  M.  Hurwitz,  R.  Meakin,  P.  Bell, 
HPCMP;  C.  Atwood) 

Computational  Research  and 
Engineering  Acquisition  Tools  and 
Environments  (CREATE)  -  Ships  Project, 
Myles  Hurwitz 

CREATE-Air  Vehicles:  A  Suite  of  Scalable 
Software  Products  for  the  Design  and 
Simulation  of  Next  Generation,  Robert 
Meakin 

CREATE  -  RF  Antenna  Group,  David  van 
Veldhuizen 

Computational  Research  and 
Engineering  Acquisition  Tools  and 
Environments  (CREATE),  David  Fisher 
and  Richard  Kendall 

Institute  for  Maneuverability  and  Terrain 
Physics  Simulation  (IMTPS),  David  A. 
Horner 

Institute  for  High  Performance 
Computing  Applications  in  Air 
Armament  (IHAAA),  Mark  Lutton 

HPC  Software  Applications  Institute 
for  Advanced  Rotorcraft  Modeling  and 
Simulation  (HI-ARMS),  Rodger  Strawn 

Institute  for  Multi-Scale  Reactive 
Modeling,  Steven  Bunte,  Brad  E.  Forch, 
Betsy  M.  Rice,  William  H.  Davis,  and 
Ernest  L.  Baker 

Battlespace  Environments  Institute: 
Building  the  Whole-Earth  Infrastructure, 
Richard  A.  Allard,  Timothy  J.  Campbell, 
Sue  Chen,  Hwai-Ping  Cheng,  and  Jing-Ru 
C.  Cheng 

Biotechnology  HPC  Software 
Applications  Institute,  Jaques  Reifman 

The  Air  Force  Research  Laboratory's 
Directed  Energy  Directorate,  Matthew 
Bettencourt 

High  Performance  Computing  Software 
Applications  Institute  for  Space 
Situational  Awareness,  Chris  Sabol,  Maj 
David  Strong,  and  Paul  Schumacher 


Space  Surveillance  Network  Analysis 
Model  (SSNAM)  SP  Integration 
Performance  Study,  Erika  Montgomery, 
Robert  Massey,  Kevin  Roe,  and  Tim 
Payne 

Mobile  Network  Modeling  Institute, 

Raju  Namburu,  Jerry  Clarke,  Ananthram 
Swami,  Brian  Rivera,  and  Monica  Farah- 
Stapleton 

Insensitive  Munitions  Portfolio,  Stanley 
DeFisher 

Education 

New  Features  Coming  to  the  OKC, 
Renee  Mullinax,  Abdul  Mohamed,  and 
John  Mason 

User  Training  in  the  User  Productivity 
Enhancement  and  Technology  Transfer 
(PET)  Program,  Susan  T.  Brown 

An  Undergraduate  Computational 
Aerodynamics  Curriculum,  Keith 
Bergeron,  Russell  Cummings,  Dave 
McDaniel,  Capt  Robert  Decker,  Maj 
Jacob  Freeman,  Capt  Charlie  Hoke, 
Jurgen  Seidel,  and  Scott  A.  Morton 

Science/Technical 

Multi-Abstraction  Levels  in  HPC: 

Enabling  Consistency,  Integration,  and 
Validation,  Thomas  D.  Gottschalk,  Dan 
M.  Davis,  and  David  R.  Pratt 

Multi-Disciplinary  Teams  for  Large-Scale, 
Massively-Parallel,  Multi-Physics  Code 
Development,  Michael  Zika 

WIS  Pacific  Regional  Hindcast,  Barbara 
Tracy  and  Deanna  Spindler 

Implementation  of  a  Real-Time  Coastal 
Ocean  Model  for  the  California  Current 
System  (RT-NCOMCCS),  Sergio  deRada, 
Stephanie  Anderson,  and  Igor  Shulman 

Predictability  of  Tropical  Cyclones  in 
Global  Atmospheric  Forecasts,  Carolyn 
Reynolds,  Justin  G.  McLay,  James  D. 
Doyle,  and  James  Goerss 

Community  Portal  for  Collaborative 
Research  on  Tsunamis,  Tom  Logan, 
Cherri  Pancake,  Dylan  Keon,  and  Craig 
Stephenson 

HPM  Simulations  of  a  Commercial 
Magnetron,  John  D.  Keisling 
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Hybrid  Partitioned  Layer  Movement 
Method  for  Fluid-Structure  Interaction, 
Young-Ho  Kim,  Jong-Eun  Kim,  Riy 
Koomullil,  and  Bharat  Soni 

Regions  of  Validity  for  the  10-Moment, 
Two  Fluid  Plasma  Model,  Robert  Lilly 
and  Uri  Shumlak 

Comparisons  of  Two-Fluid  Plasma 
Models,  Bhuvana  Srinivasan  and  Uri 
Shumlak 

Extending  Automated  Meshing  to  Large 
Multi-Block  Domains:  Countermine 
Mesh  Generation  on  HPC  Platforms, 
Owen  J.  Eslinger  and  Amanda  M.  Hines 

Incorporation  of  Detonation  Shock 
Dynamics  into  CTH,  David  Littlefield  and 
Young-Ho  Kim 

Energetics  and  Warheads  Modeling 
and  Simulation,  Chuck  L.  Chin,  Stanley 
DeFisher,  and  Ernie  Baker 

Modeling,  Inference,  and  Analysis 
of  Complex  Nonlinear  Interaction 
Networks,  W.  Garrett  Jenkinson  and 
John  Goutsias 

Parallel  Implementation  of  Certain 
Robust  Regression  Methods  Using  Lazy 
Evaluation  in  Python,  Alan  Chalker 

Some  Comparative  Benchmarks 
for  Linear  Algebra  Computations  in 
MATLAB  and  Scientific  Python,  Jose 
Unpingco 

SSH  Toolbox  for  MATLAB:  Point  and 
Click  HPC  Access  From  the  Desktop, 
Siddharth  Samsi,  John  Nehrbass, 
Harrison  B.  Smith,  Tanner  Suttles,  and 
Andrew  Warnock 

A  Scalability  Study  on  Multicore  Cluster 
Systems  of  an  AFRL  Radar  Frequency 
Tomography  Imaging  Code  Written  in 
MATLAB®  for  Parallel  Execution  Using 
Star-P®,  Bracy  H.  Elton  and  Kevin  M. 
Magde 

DREN  Internet  Protocol  version  6  (IPv6) 
Support  for  HPC,  John  M.  Baird  and  Ron 
Broersma 

Comparison  of  Turbo  Decoder  and 
Packet  Acquisition  Error  Rates  in 
Frequency-Hop  Spread-Spectrum 
Systems,  Everest  W.  Huang  and 
Frederick  J.  Block,  IV 

Compression  of  Network  Packet 
Traces  to  Facilitate  Sharing  of  Security 
Information,  Jean  Blaudeau  and  William 
Yurcik 


Metadata-Based  Network  Traffic 
Database  Management  System  Model, 
Keesook  J.  Han,  Matthew  Bentz,  Chales 
George,  Christopher  Hall,  Ross  Marullo, 
and  John  C.  Kieffer 

Structure  and  Dynamics  of  Liquid 
Squalane  on  a  Solid  Surface,  Meslin 
Tsige  and  Soumya  S.  Patnaik 

Molecular  Dynamics  Simulation  of 
the  Kinetic  Reaction  of  Ni  and  Al 
Nanoparticles,  Brian  J.  Henz,  Takumi 
Hawa,  and  Michael  R.  Zachariah 

LAMMPS  Simulations  of  the  Effects  of 
Chirality  and  Diameter  on  Pullout  Force 
in  a  Carbon  Nanotube  Bundle,  Dustin 
Majure,  Richard  Haskins,  Robert  Maier, 
Nicholas  J.  Lee,  Robert  M.  Ebeling, 
Charles  P.  Marsh,  Anthony  Bednar,  and 
Charles  C.  Cornwell 

Novel  Mechanism  for  the  Dissociation 
of  H20  and  the  Diffusion  of  O  and 
H  Along  the  aAl2Os  (0001 )  Surface, 
Jennifer  Synowczynski,  Jan  Andzelm, 
and  Dionisios  Vlachos 

Calculation  of  Substituent  Effects  on  CH 
Acidity  of  CB11  HI 2-,  Jaroslav  Vacek, 
Jana  Chocholousova,  and  Josef  Michl 

Targeting  Jitter:  Identification  and 
Reduction  of  Excess  Operating  System 
Noise  on  Commodity  Cluster,  Don  Bahls 
and  Oralee  Nudson 

Accelerated  Design  Space  Search 
Techniques  for  Computer  Experiments 
on  High  Performance  Computers, 
Aytekin  Gel,  Allan  Grosvenor,  Esma  S. 
Gel,  and  Murat  Kulahci 

Improving  Parallel  Code  Performance 
for  HPC  Systems  with  Multi-Core 
Processors,  Scott  Spetka,  Scot  Tucker, 
Susan  Emeny,  Dennis  Fitzgerald,  George 
Ramseyer,  and  Richard  Linderman 

Technique  for  Translation  of  Bullet  in- 
Flight  Data,  Susan  Neczyporuk 

SceneGen:  Defining  Terrain-based 
Scenes  for  Engineering  Analysis,  Barry 
C.  White 

Improved  Parallel  3D  FDTD  Simulator 
for  Photonic  Crystals,  Jason  S.  Ayubi- 
Moak,  Stephen  M.  Goodnick,  Dan 
Stanzione,  Gil  Speyer,  and  Paul  Sotirelis 

Evaluation  of  Software  Packages  at 
the  ARL-MSRC  for  CCSD(T)  Quantum 
Calculations,  James  lanni 


HPCMP  Sustained  Systems  Performance 
Test,  Paul  M.  Bennett 

Enabling  Computational  Technologies 
for  High  Performance  Computing,  The 
Center  for  Applied  Scientific  Computing, 
Lori  Diachin 

Compiler  Comparisons  on  ARSC 
Resources,  Edward  A.  Kornkven,  Don 
Bahls,  and  Tom  Logan 

DREN  2008  Networking 
and  Security  Conference 

Presentations 

CAC  Enabling  Your  Website,  Lee 
Hinman  and  Jay  Kline 

Carriers  of  Carriers,  Ben  Eater,  Juniper 
Networks,  Federal 

Changing  the  Game  -  National  and  DoD 
Science  and  Technology  Initiatives  to 
Secure  Cyberspace,  Steven  King 

Data  Protection,  Tim  Yeager 

Defense  Research  Engineering  Network 
(DREN)  Resources  Used  to  Disseminate 
High  Resolution  Displays  for  Missile 
Defense  Agency  (MDA)  Research 
Development  Test  and  Evaluation 
(RDT&E)  Events,  Ray  May 

Distributed  Data  Logging  for  Analysis, 
Detection,  and  Prevention  of  Network 
Attacks,  John  J.  Tran  and  Ke-Thia  Yao 

DREN  CERT/CNDSP,  Jeremy  Batson  and 
Joe  Molnar 

DREN  Vision,  Walt  Williams 

Effective  Security  Operations 
Management,  Sunil  Bhargava,  CTO, 
Intellitactics,  Inc. 

HPC  CERT  South  Build  Out,  Tony 
George 

HPCMP  Return  on  Investment,  Danny 
Weddle 

HPCMP  Storage  Initiatives,  Ralph 
McEldowney 

HPWREN  Wireless  Research  Network, 
Hans-Werner  Braun,  UCSD 

Internet2  Network:  Developing  and 
Deploying  a  Hybrid  Optical  and  Packet 
Network,  Christian  Todorov 
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IPv6,  Ron  Broersma 
JMETC,  Frank  Barone 

Mission  Critical  Communications  -  The 
Role  of  Video  in  Next  Generation  Video 
Communication  for  the  Warfighter, 
Michael  Coyne 

New  IDS  Strategies  and  Interrogator, 
Jason  Schaum 

Outreach,  Stephen  Bowman 
Overview  of  DoD  Identity,  Joe  Molnar 

PKlinit  in  the  HPCMP  (Ditching  Your 
SecurlD  Card),  Ken  Renard 


Secure  Wireless,  Rob  Scott  and  Stephen 
Bowman 

Securing  Today's  Networks,  Rich 
Whittney,  Juniper  Networks,  Federal 

State  of  DREN,  Mark  Heck  and  Paul 
Savercool 

The  Identification  and  Demonstration  of 
Advanced  Applications  and  Capabilities 
for  the  Department  of  Veterans  Affairs, 
Steven  Pirzchalski,  Department  of 
Veteran  Affairs 


Threat  Cell  Presentation  -  HPC  CERT 
North,  Tim  Dunn 

Use  of  Operations  Security  (OPSEC) 
Procedures  to  Safeguard  DoD  Networks 
and  Private  Industry's  Research  Data, 
Joyce  Schulte 
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Acronyms 

2D 

Two-Dimensional 

3D 

Three-Dimensional 

ABL 

Airborne  Laser 

ACAD 

Automated  Computer  Aided  Design 

ACM 

Association  for  Computing  Machinery 

ADA 

Atmospheric  Decision  Aid 

ADC 

Allocated  Distributed  Center 

AEDC 

Arnold  Engineering  Development  Center 

AF 

Air  Force 

AFB 

Air  Force  Base 

AFDD 

Aeroflightdynamics  Directorate 

AFFTC 

Air  Force  Flight  Test  Center 

AFGROW 

Air  Force  Crack  Growth  Program 

AFOSR 

Air  Force  Office  of  Scientific  Research 

AFRL 

Air  Force  Research  Laboratory 

AFRL/RB 

Air  Force  Research  Laboratory,  Air  Vehicles 

Directorate 

AFRL/RD 

Air  Force  Research  Laboratory,  Directed  Energy 

Directorate 

AFRL/RH 

Air  Force  Research  Laboratory,  Human 
Effectiveness  Directorate 

AFRL/R1 

Air  Force  Research  Laboratory,  Information 

Directorate 

AFRL/RV 

Air  Force  Research  Laboratory,  Space  Vehicles 

Directorate 

AFRL/RW 

Air  Force  Research  Laboratory,  Munitions 

Directorate 

AFRL/RX 

Air  Force  Research  Laboratory,  Materials  and 
Manufacturing  Directorate 

AFRL/RY 

Air  Force  Research  Laboratory,  Sensors 

Directorate 

AFRL/RZSP 

Air  Force  Research  Laboratory,  Space  and 
Missile  Directorate,  Propulsion  Division 

AFSEO 

Air  Force  SEEK  EAGLE  Office 

AFWA 

Air  Force  Weather  Agency 

AHPCRC 

Army  High  Performance  Computing  Research 

Center 

AMCOM 

Army,  Aviation  and  Missile  Command 

AMOS 

Air  Force  Maui  Optical  and  Supercomputing 

Site 

AMRDEC 

Army  Aviation  and  Missile  Research 
Development  and  Engineering  Center 

AMSAA 

Army  Material  System  Analysis  Agency 

APB  Acquisition  Program  Baseline 

API  Application  Program  Interface 

ARC  Affiliated  Resource  Center 

ARDEC  Armament  Research  Development  and 

Engineering  Center 

ARL  Army  Research  Laboratory 

ARL/WMRD  Army  Research  Laboratory  Weapons  and 
Materials  Research  Directorate 


ARO  Army  Research  Office 

ARSC  Arctic  Region  Supercomputing  Center 

ASD(NII)  Assistant  Secretary  of  Defense,  Networks  and 

Information  Integration 

ATFL1R  Advanced  Targeting  Forward  Looking  Infrared 

ATR  Automatic  Target  Recognition 

AV  Air  Vehicles 


BEI  Battlespace  Environments  Institute 

BFI  Bullet  and  Fragment  Impacts 

BHSAI  Biotechnology  HPC  Software  Applications 

Institute 

BoD  Board  of  Directors 


C4I  Command,  Control,  Communications, 

Computers  and  Intelligence 
C4ISR  Command,  Control,  Communications, 

Computers,  Intelligence,  Surveillance  and 
Reconnaissance 

CAP  Capability  Applications  Project 

CBE  Computationally  Based  Engineering 

CCAC  Consolidated  Customer  Assistance  Center 

CCD  Charge  Coupled  Device 

CCM  Computational  Chemistry  and  Materials  Science 

CDLT  Collaborative  and  Distance  Learning 

Technologies 

CE  Computational  Environment 

CEA  Computation  Electromagnetic  and  Acoustics 

CECOM  Communications  Electronics  Command 

CEM  Computational  Electromagnetic 

CERT  Computer  Emergency  Response  Team 

CFD  Computational  Fluid  Dynamics 

CID  Combat  Identification 

CNDSP  Computer  Network  Defense  Service  Provider 

CNMOC  Commander,  Naval  Oceanography  and 

Meteorological  Center 
COOP  Continuous  Operations  Plan 

CPU  Central  Processing  Unit 
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CREATE 

Computational  Research  and  Engineering 
Acquisition  Tools  and  Environments 

CSA 

Comprehensive  Security  Assessment 

CSB 

Configuration  Steering  Board 

CSE 

Computational  Support  Environment 

CSM 

Computational  Structural  Mechanics 

CST 

Collaborative  Simulation  and  Testing 

CTA 

Computational  Technology  Areas 

C-USS 

Customer-User  Satisfaction  Survey 

CWO 

Climate/Weather/Ocean  Modeling  and 
Simulation 

DAAC 

DARPA 

DCAA 

DDG 

DDR&E 

DE 

DFT 

DH  PI 

DNS 

DoD 

DOE 

DOT&E 

DPG 

DREN 

DSRC 

DSYMAS 

DT&E 

DTO 

DTRA 

DUSD(S&T) 


Data  Analysis  and  Assessment  Center 

Defense  Advanced  Research  Projects  Agency 

Defense  Contract  Audit  Agency 

Guided  Missile  Destroyer 

Director  of  Defense  Research  and  Engineering 

Directed  Energy 

Density  Functional  Theory 

Dedicated  HPC  Project  Investment 

Direct  Numerical  Simulations 

Department  of  Defense 

Department  of  Energy 

Director,  Operational  Test  and  Evaluation 

Dugway  Proving  Ground 

Defense  Research  and  Engineering  Network 

DoD  Supercomputing  Resource  Center 

Dynamic  Systems  Mechanics-Advanced 

Simulation 

Developmental  Test  and  Evaluation 
Defense  Technology  Objectives 
Defense  Threat  Reduction  Agency 
Deputy  Under  Secretary  of  Defense  (Science 
and  Technology) 


EM  Electromagnetic 

EMC  Electromagnetic  Compatibility 

EMI  Electromagnetic  Interference 

EMS  Energetic  Materials 

ENS  Electronics,  Networking,  and  Systems/C4l 

EOTC  Education,  Outreach,  and  Training  Coordination 

EQM  Environmental  Quality  Modeling  and  Simulation 

ERDC  Engineer  Research  and  Development  Center 

ESM  Enterprise  Security  Management 

ESMF  Earth  System  Modeling  Framework 

ET  Enabling  Technologies 


FCS 

Future  Combat  Systems 

FDTD 

Finite  Difference  Time  Domain 

FEM 

Finite  Element  Method 

FFT 

Fast  Fourier  Transform 

FL1R 

Forward  Looking  Infrared 

FLOPS 

FLoating-point  Operations  Per  Second 

FMS 

Forces  Modeling  and  Simulation 

FY 

Fiscal  Year 

GB 

Gigabytes 

GCCS 

Global  Command  and  Control  System 

GF 

Gigaflop 

GIG 

Global  Information  Grid 

GPS 

Global  Positioning  System 

GSA 

General  Services  Administration 

GWOT 

Global  War  On  Terror 

HBCUs 

Historically  Black  Colleges  and  Universities 

HI-ARMS 

HPC  Institute  for  Advanced  Rotorcraft  Modeling 
and  Simulation 

HPAC 

Hazard  Prediction  and  Assessment  Capability 

HPC 

High  Performance  Computing  or  High 
Performance  Computer 

HPCAP 

High  Performance  Computing  Advisory  Panel 

HPCC 

High  Performance  Computing  Centers 

HPCMP 

High  Performance  Computing  Modernization 
Program 

HPCMPO 

High  Performance  Computing  Modernization 
Program  Office 

HPM 

High  Power  Microwave 

HPTi 

High  Performance  Technologies,  Incorporated 

HYCOM 

Hybrid  Coordinate  Ocean  Model 

ICD 

Initial  Capabilities  Document 

IHAAA 

Institute  for  HPC  Applications  to  Air  Armament 

IED 

Improvised  Explosive  Devices 

IEEE 

Institute  of  Electrical  and  Electronics  Engineers 

IHDE 

Integrated  Hydrodynamics  Design  Environment 

ILM 

Information  Lifecycle  Management 

IM 

Insensitive  Munitions 

IMT 

Integrated  Modeling  and  Test  Environments 

IMTPS 

Institute  for  Maneuverability  and  Terrain  Physics 
Simulation 

I/O 

Input/Output 

IP 

Internet  Protocol 

l-SSA 

Institute  for  Space  Situational  Awareness 
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JEOM 

Joint  Educational  Opportunities  for  Minorities 

JET 

Joint  Engineering  Team 

JFCOM 

Joint  Forces  Command 

JHL 

Joint  Heavy  Lifting 

JITC 

Joint  Interoperability  Test  Command 

JML 

Joint  Model  Library 

JSF 

Joint  Strike  Fighter 

J-UCAS 

Joint  Unmanned  Combat  Air  Systems 

JWCO 

Joint  Warfighting  Capability  Objective 

KE 

Kinetic  Energy 

KEI 

Kinetic  Energy  Interceptor 

LAN 

Local  Area  Network 

LCO 

Limit  Cycle  Oscillation 

LCS 

Littoral  Combat  Ship 

LEAPS 

Leading  Edge  Advanced  Prototyping  for  Systems 

LES 

Large  Eddy  Simulations 

LHA 

Amphibious  Assault  Ship-Tarawa  Class 

LHD 

Amphibious  Assault  Ship-Wasp  Class 

LLNL 

Lawrence  Livermore  National  Laboratory 

LSN 

Large  Scale  Network 

M&S 

Modeling  and  Simulation 

MAGIC 

Meshing  and  Geometry  Innovation 

Collaboration 

MAV 

Micro  Air  Vehicles 

MDA 

Missile  Defense  Agency 

MDAP 

Major  Defense  Acquisition  Program 

MFT 

Multiphase  Flow  Target  Response 

MFS 

Manned  Flight  Simulator 

MHD 

Magnetohydrodynamics 

MHPCC 

Maui  High  Performance  Computing  Center 

MIT 

Massachusetts  Institute  of  Technology 

MNMI 

Mobile  Network  Modeling  Institute 

MOA 

Memorandum  of  Agreement 

MOS 

Mississippi  and  Ohio  State  University 

MPI 

Message  Passing  Interface 

MRAP 

Mine  Resistant  Ambush  Protected 

MSI 

Minority  Serving  Institution 

MSRC 

Major  Shared  Resource  Center 

MSRM-IM 

Multi-Scale  Reactive  Modeling  and  Simulation 
of  Insensitive  Munitions 

MTBF 

Mean  Time  Between  Failures 

NASA 

National  Aeronautics  and  Space  Administration 

NAVAIR 

Naval  Air  Systems  Command 

NAVO 

Naval  Oceanographic  Office 

NAVSEA 

Naval  Sea  Systems  Command 

NAWC 

Naval  Air  Warfare  Center 

NCA&T 

North  Carolina  Agricultural  and  Technical 
University 

NDSEG 

National  Defense  Science  and  Engineering 

NDU 

National  Defense  University 

NGTS 

Next  Generation  Technical  Services 

NOC 

Network  Operation  Center 

NOGAPS 

Navy  Operational  Global  Atmospheric 

Prediction  System 

NRL 

Naval  Research  Laboratory 

NRT 

Near  Real  Time 

NRTC 

Near  Real  Time  Computing 

NSF 

National  Science  Foundation 

NSWC 

Naval  Surface  Warfare  Center 

NUWC 

Naval  Undersea  Warfare  Center 

NVL 

Night  Vision  Laboratory 

NVRAM 

Non-Volatile  Random  Access  Memory 

NWDC 

Navy  Warfare  Development  Command 

oc 

Optical  Carrier 

OIPT 

Overarching  Integrated  Product  Team 

OKC 

Online  Knowledge  Center 

ONR 

Office  of  Naval  Research 

OPFOR 

Opposing  Force 

OSD 

Office  of  Secretary  of  Defense 

PE 

Predicting  Probability  of  Effect 

PEO 

Program  Executive  Office 

PET 

User  Productivity  Enhancement  and  Technology 
Transfer 

PEUO 

Physics-Based  Environment  for  Urban 

Operations 

PIE 

Portal  to  the  Information  Environment 

Pk 

Predicting  Probability  of  Kill 

PKI 

Public  Key  Infrastructure 

RBD 

Rigid  Body  Dynamic 

RDT&E 

Research,  Development,  Test  and  Evaluation 

RF 

Radio  Frequency 

RFP 

Request  For  Proposals 

RMWIM 

Rocket  Motor  and  Warhead  Impact  Modeling 

ROI 

Return  On  Investment 
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RT 

Real-Time 

RTC 

Real-Time  Computing 

RTTC 

Redstone  Technical  Test  Center 

S/AAA 

Service/ Agency  Approval  Authority 

S&T 

Science  and  Technology 

SABL 

Stable  Atmospheric  Boundary  Layer 

SAD 

Search  and  Determine 

SAS 

Software  Applications  Support 

SC 

Supercomputing 

SDP 

Service  Delivery  Points 

SDREN 

Secret  Defense  Research  and  Engineering 
Network 

S1F 

Schools  Interoperability  Framework 

SIP 

Signal/lmage  Processing 

SLM 

Storage  Lifecycle  Management 

SMDC 

Space  and  Missile  Defense  Command 

SMCCOM 

Soldier  Biological  Chemical  Command  Research 
Development  Engineering  Center 

SMDC 

Space  and  Missile  Defense  Command 

SOF 

Special  Operations  Force 

SOP 

Standard  Operating  Procedure 

SPAWAR 

Space  and  Naval  Warfare  Systems  Command 

SPI 

Software  Protection  Initiative 

SSA 

Space  Situational  Awareness 

SSN 

Space  Surveillance  Network 

STEM 

Science,  Technology,  Engineering,  and 

Mathematics 

STOVE 

Short  Take-Off  and  Vertical  Landing 

T&E 

Test  and  Evaluation 

TACOM 

Tank-Automotive  Command 

TARDEC 

Tank-Automotive  Research,  Development  and 
Engineering  Center 

TB 

Terabytes 

Tl 

Technology  Insertion 

TPD 

Technical  Planning  Document 

UAV 

Unmanned  Air  Vehicles 

UCAS 

Unmanned  Combat  Air  Systems 

UCAV 

Unmanned  Combat  Air  Vehicles 

UGC 

Users  Group  Conference 

UIT 

User  Interface  Toolkit 

UNAfold 

Unified  Nucleic  Acid  Folding 

US 

United  States 

USACE 

US  Army  Corps  of  Engineers 

USAFA 

US  Air  Force  Academy 

USAMRIID 

US  Army  Medical  Research  Institute  of 
Infectious  Diseases 

USAMRMC 

US  Army  Medical  Research  and  Materiel 
Command 

USD(AT&L) 

Under  Secretary  of  Defense  for  Acquisition 
Technology  and  Logistics 

UTC 

User  Training  Coordination 

VED 

Virtual  Electromagnetic  Design 

VSIPL 

Vector  Signal  Image  Processing  Library 

VT 

Virtual  Token 

WAN 

Wide  Area  Network 
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Opportunities  to  Use  HPCMP  Resources 


Any  researcher  or  developer  who  is  actively  supporting  the  Department  of  Defense  in  the  areas  of  R&D  and/or  T&E 
may  have  access  to  HPCMP  resources.  This  includes  potential  users  who  are  government  employees,  government 
contractors,  and  researchers  from  the  academic  community. 

To  get  started,  you  must  sign  a  DoD  Information  Systems  User  Agreement  and  take  the  following  DoD  training: 

Information  Assurance  Awareness  Training 
Personal  Electronic  Devices/Removable  Storage  Media 
Phishing  Awareness 

Next,  apply  for  an  account  via  the  Portal  to  the  Information  Environment  (plE)  at  https://ieapp.erdc.hpc.mil/info/ 
userRequestHome.jsp. 

For  assistance,  contact  your  Service/Agency  Approval  Authority  (S/AAA).  Each  government  organization  within  the 
HPCMP  community  has  an  assigned  S/AAA  who  is  responsible  for  setting  up  new  accounts;  modifying  new  accounts; 
creating  projects  in  plE;  allocating  resources  to  subprojects;  and  approving  project  requirement  questionnaires. 
Locate  your  S/AAA  in  plE  (under  "Report  Information  Environment"  or  at  https://www.hpcmo.hpc.mil/bridge/). 
Your  S/AAA  approves  your  account  request  and  assists  in  its  activation.  A  National  Agency  Check  (NAC)  may  be 
required. 

You  can  work  with  your  S/AAA  to  establish  a  new  project  in  plE,  complete  a  requirements  questionnaire,  and  access 
HPCMP  resources  atone  or  more  of  the  DSRCs.  Allocation  of  resources  are  made  by  the  Service  organization  and 
not  by  the  HPCMP  Office. 


